Discussion regarding the complex response structure for REST & GraphQL (Developer Experience)

Discussions
21

In addition to my github issue and after reading carefully the whole discussion, I have notice some points which make me (ironically) laughing:

  • “At this time we are not prepared or planning to modify this data structure in the response, the time for that type of modification has passed as it should have been addressed during the RFC process. We do understand that it was not clear at the time during our RFC process this was the case and we are already doing retrospectives to improve this process.” => so the priority should be on fixing the issue instead of speaking about how to improve for next time (no worries, you will have months or even years for that), especially since “next time” is somewhere, far away, in the futur.

  • “The primary goal behind this new structure was to pick some kind of standard for a response structure, ideally not reinvent the wheel and try to use an already existing standard” => good goal! We are lucky, there is already an existing javascript reference implementation named “Apollo”, it’s even written black on white in the github of the project. Unfortunately, strapi v4 is incompatible with that standard (caching issue). At the end, it seems that strapi v4 is just creating the last (15) competing standards… congrats…

  • “In our case we picked JSON API: https://jsonapi.org/” => Good choice for REST API, not at all for GraphQL which is different by design and have nothing to do with JSON API

  • “This meant unifying the REST and GraphQL data responses, error handling, filtering/paging system, ect.” => again, there is nothing to unify, REST and GraphQL are different by design and even it’s the reason why GraphQL was created. Going against that difference means missing the primary goal of GraphQL. Sorry but if strapi goal is to unify GraphQL with REST then the only way is to drop GraphQL support

  • “Remove Attributes and throw everything under data in GraphQL => Not possible due to breaking changes” => I guess it’s a joke?!? Seriously guys, who is using strapi v4 in production right now? Strapi v4 is more a beta than a stable product atm and only early adopters are trying to migrate on it (no offense, it’s a perfectly normal situation). Even the migration guide is missing for now! If your concern is about semver, a very easy solution is to release a new graphql plugin (for example “graphql-apollo-plugin”) with an initial v4.0.6 version and that’s it! Simple, easy. Once v5 will be ready, simply drop the current graphql-plugin and replace it with graphql-apollo-plugin (renamed graphql-plugin, ofc)


Now let me provide a bit of context:
I’m currently working on a huge migration both frontend (vue v2.x => v3.x) and backend (strapi v3.x => v4.x) with a strong deadline of mid-feb. I have already spent a lot of time to migrate DB schema from v3 to v4 manually.

Now, no doubt, GraphQL API in v4 is just unusable as is and so my mid-feb release is fucked because I will have to switch back on strapi v3 everything which is already migrated at frontend side.

Not only that, but since strapi v3 is in maintenance mode, I will have to plan another migration for backend side (aka building my own or switching on another strapi-like product) so it’s the whole roadmap which is now totally fucked.

Tomorrow, I will have to explain that to my boss and, if I’m not fired immediately, I will have to buy a lot of redbull, in order to develop 24/7 during the next months. I guess everyone here could very well understand why I’m really upset.

I’m really sorry if my words aren’t as kind as they should be, I’m just very disappointed because of that horrible situation. However, I’m still confident about a brillant futur for strapi and love all guys which works hard to make it better.

10 Likes
22

After using Strapi v4 for a month, I think the same… It’s more of a beta than a stable product.
I don’t know, but I don’t think anyone is using Strapi v4 in production.

And I totally agree with that REST and GraphQL are different by design.

Please prioritize fixing GraphQL’s answer :pray:

3 Likes
23

yes - this approach along w/ no fixes in strapi 3 shows it isn’t ready for production.

1 Like
24

I am no one to complain but this deeply nested data/attributes is a deal breaker for me. Seriously what problem it was trying to solve in the first place?

6 Likes
25

I created a simple middleware to transform V4’s REST responses into a better format. Basically remove the existence of data and attributes
Hope this helps.

7 Likes
26

Interesting. Is there a way to do the same on graphql API?

27

nice! - playing around (also using graphql) but would want to exclude /api/auth/local and /api/users/me

28

As a tiny update to this, we have scheduled an internal meeting to discuss the issue at length on tuesday. We are taking into account all of the feedback gathered here and elsewhere.

When I have more information to share on this, I will :slight_smile:

Please do feel free to keep the feedback coming.

6 Likes
29

encouraging - thanks for updating that it is being looked at. I think even an interim step of a middleware that reverts to the ‘old’ format would help with those of us porting. longer term solutions will no doubt require more thought.

31

I was in the middle of a v3 → v4 migration when I saw those data and attributes pollute my graphQL queries.

I’ve created an account just to mention that I share the opinion of the folks above, and I’m really hoping to be able to use the old, intuitive and logical structure of the GraphQL responses.

3 Likes
32

So I was really bothered by this and wrote a normalizer on the front end that gets rid of both data and attributes, and also handles Typescript.

I’ve included an example of the normalization in the comments

It’s a very very rough draft. I need to clean it up, there are a few edge cases it doesn’t handle, and it is a bit too aggressive about stripping, but I’m wondering if it would be of use to anyone?

33

Was also quite excited about the v4 release until I ran across this little detail. Seems like Apollo Federation is also broken with this release, not sure if it is related to the same schema changes. Just a small datapoint in the bucket, but will be reverting back to v3 until there is some resolution here.

34

Hi @DMehaffy ,

Thanks a lot for your last update. Unless you (as strapi dev team) decide to stay locked in the room until providing a fix, I guess the meeting is over. Any news?

The more the comeback arrives here, the clearer the community needs became: a “breaking” change which get rides of useless data wrapper for single asset query response and no id and attributes split anymore. Really hope a pragmatic decision was taken today :pray: :candle:

I love strapi and I don’t want to be forced to switch on another solution… :cry:

35

Yup I was just re-reading through our notes from the meeting we did this morning and intended to share what we discussed and what we are thinking.

Actions we are taking right now:

  • Doing a bit more research into other existing implementations such as Facebooks Relay: https://relay.dev/ and how they are doing things as functionally our setup is very similar to theirs
  • Researching a bit into getting caching working (it is possible but not just as simple as apollo’s native config for it) as it’s a huge area of concern from everyone

Actions we plan to take in the next month or so:

  • Have a community call (or steal part of an existing one) to cover some of the background with this
  • Record a video or write a lengthy blog-post explaining some of the future needs we have from the response structure to support the following three features:
    • Content Versioning
    • Publication Workflows
    • Better meta information about an entity’s i18n locale
  • Breaking down our focus towards keeping things stable and some bits of the longer term plan

What we discussed was the following (keep in mind we aren’t taking a full decision right now because some people are on vacation next week and we want to do more research before we just rush into things).
I hope everyone can understand we don’t want to just take a decision on this lightly, especially now that we have extended the v3 EOL til Sept 2022.

  • Removing data {} is not possible now or in the future because it would conflict with high level meta {}
  • attributes {} could be removed in theory now but it would conflict with our long term plans in the future with meta {} (merged, we will probably touch on this in the video/blog post)
  • We are willing to make a breaking change for GraphQL plugin but would require a long and complex RFC before we would do that, not a short term solution but yes a plugin could technically be ejected from the versioning system the core uses. There would need to be a lot of validation from the community around the RFC that we didn’t have when we first issued the RFC (largely the problem on our side because it wasn’t clear what feedback we wanted, and it wasn’t shared enough)
  • Researching Relay => why does it work so well => why is it so accepted by their community => what learnings can we take from that => their caching layer?
  • Can SDKs help the end user? Is this stuff we can build to help?

We also discussed a bit on the RFC side and what went wrong and how to prevent it in the future:

  • RFC needs more readers AND interactions (comments, discussions, debates, not preferences but breaks downs on why an alternative is better with trade offs, ect)
  • More data in the RFC, too much internal information and context wasn’t shared
  • RFC format wasn’t clear enough, not enough descriptions
  • RFC process was too fast // Strapi v4 Beta process was too short (won’t happen again, we were under a time crunch due to poor timeline planning in v4)
  • Ideally we really should publish RFCs long before we start development and engineering team needs to be more active reading responses and responding to RFCs. Small team problems that we are fixing slowly but simply we need more devs.

Funnily enough one of the original respondents to the RFC when we published it was a community member who now works for us so we were able to rapidly gather some cool insights and in the future we would like to have some of these discussions in person such as in community calls going forward as it was really helpful to get that point of view.

TLDR: Nothing major happening right now, it’s clear we need to do some research but we need to make decisions carefully, we can’t “just revert” back to v3 style. Keep in mind that we did what we did as we were building pre-reqs for what we want to build in the future (ideally this year). That was never described to any of you and it’s what we need to do now.

Give us a bit of time we are listening and we do plan to engage with the community to find a working resolution to this. Your opinions are very valuable to us, time crunches just didn’t make things easy and too much got rushed.

5 Likes
36

on Relay - their response is as follows form their docs or am I missing re. the comparison

{
  "data": {
    "user": {
      "id": "4",
      "name": "Mark Zuckerberg",
      "username": "zuck"
    },
    "viewer": {
      "actor": {
        "name": "Your Name"
      }
    }
  }
}
2 Likes
37

I second this. There is no redundancy in their data structure, so it’s no surprise to me that’s “accepted by their community” as @DMehaffy asked.
IMO, it has nothing to do with adding the same two properties to each nested object.

To me, using GraphQL with Strapi was all about simplicity and flexibility. Now I’ll have to implement middlewares to recursively flatten all my objects, which is bad for both readability and performance.

I feel it’s going against simple dev principles like DRY, YAGNI, BDUF, but I’m sure I don’t have the full scope and vision, that’s just how I feel for my simple use cases.

4 Likes
38

Very interesting to see all this in the roadmap and the team being so proactive and receptive of feedback. T H A N K S ! !

As a tech lead who led adoption of GraphQL in a mid-sized company I would recommend to not overlook what Apollo is doing. In particular, I’d like to recommend this article from Sashko Stubailo (one of the core engineers of Apollo V1) who is a very respected voice in the GraphQL ecosystem: Understanding pagination: REST, GraphQL, and Relay - Apollo GraphQL Blog It makes a great case illustrating pagination issues and the trade-offs around conventions. That was a great kick-off for internal discussion in our experience when we needed to decide for Apollo or Relay. We are pretty happy with the flexibility Apollo provides.

An extra mention worth doing is Apollo has always been a DX-first company (they did Meteor before) and it seems that taking their adoption strategy should guarantee a good response from devs who interact with Strapi using GraphQL. They are also widely adopted so a big part of the GraphQL community will just get what they expect.

Looking forward for more interaction with the community :slight_smile:

1 Like
39

Dear Strapi team,

Why so many ‘data/attributes’ flying around?

I really like Strapi (there are so many more pros than cons) but have been struggling with your recent changes in v4 related to the ‘Unified response format,’ even wondering if there might be an alternate headless solution that might prove to be more effective.

If one is trying to consume a single endpoint that has 2 or 3 levels hierarchy, which easily occurs when one defines any relations, media fields, components, or dynamic zones, it seems likely that the developer (who uses Strapi to build the hierarchy) would be the one responsible for any necessary meta or pagination that runs so deeply within the model they’re constructing, making the data/attribute wrapping redundant for a majority of the use cases beyond the 2nd level.

In your documentation, you say a single entry returns an object with id, attributes, and meta. This seems logical for level 1 ( of which I would be in accord), as I would consider the endpoint to be one query (regardless of filtering parameters); any subsequent level queries are not directly part of the interface.

As an initial test, I have spent some time creating a transformation layer to remove this wrapping (in one instance, trimming by 4 data/attribute levels), but have yet to come up with a standard and clean approach to use across all my models.

In my opinion, I shouldn’t really need to create a transformation layer specifically to eliminate what in all current practices feels like an arbitrary rule for distant future-proofing.

I’ve considered several solutions: 1) restrict my models to a single dynamic zone across one level (which isn’t ideal in all cases for data-entry, and defeats the flexibility of Strapi’s types), thus keeping the response flatter; 2) create a middleware service to transform the response (as others have suggested), but this seems too cumbersome and not the best place to perform such a transformation; 3) create a plugin to handle transformations, which is not ideal but would in theory be helpful for migrations; or 4) find another headless CMS with a cleaner response format.

I would hate to have gone into production with v3 (the version and response structure that originally won me over) and be faced with adding a transformation layer to migrate to v4.

The following would be ideal for dev-friendly front-end consumption:

Apps should be able map the object with ease without transformation:
‘page.content.type’ instead of ‘data.attributes.page.content.type.data.attributes.name.’

For instance, with a Vue component, passing data attributes from the response should map sensibly without the front-end dev having to unnecessarily transform the object to remove verbosity (at least, that is what I wish to provide her/him):

<component v-bind:is="$compName(attrs.__component)" :attributes="attrs" />

instead of;

<component 
v-bind:is="$compName(data.attributes.page.content.__component)" 
:attributes="data.attributes.page.content.attrs" 
/>

I hope you will at least consider removing the data/attributes wrapper (from both REST and GraphQL APIs) on nested objects that do not require meta/pagination, or include them only when directly called by an endpoint return one level. I hope this feedback is helpful.

10 Likes
40

Hello good,

Thanks for the work done, but I want to contribute that with version 4 converting a json to dart, go or similar is simply frustrating, you have to be doing touch-ups by hand and not a few.

Thanks.

41

Thanks a lot pal, exactly what I needed. It’s so relaxing to see recursion doing amazing stuff.
Kudos!

1 Like