Discussion regarding the complex response structure for REST & GraphQL (Developer Experience)

Sorry, I don’t understand, didn’t I cover that in my response? I continue to use Strapi and I am working around the current state of things by custom mapping and generating custom types (my project is entirely in typescript).

how do you manage all the sh*** in the GraphQL response?

I gave up. I’m using REST API. I did look at GraphQL as a fix for things, but since mapping/flatten the output is a violation of GraphQL spec, I couldn’t see a way of doing it that was any better than the approach I’d half-done with my rest implementation, so I’ve gone back to that.

Thank you for your feedback. The Strapi team’s goal is to ensure that we continue to create the best possible CMS we can.

Changes, like going from Srapi v3 to v4, brought some new challenges. But we are listening to the community, even when some comments are negative.

We are also human beings, and we are working at a company where we believe in the product and want to continue making it better.

We will be holding a Livestream around Strapi v3 vs. v4 to discuss some changes made and why. Also, it would be a great place to ask questions.

You can join us here Difference Between Strapi v3 and v4, what changed, and why?

I know the migration process was difficult, specifically when migrating a v3 GraphQl Strapoi Project to v4 since we fundamentally rewrote how that is handled in version 4.

But if you have any suggestions to help us improve the process. Please feel free to let me know.

Thanks, Toby.

I know changes can be frustrating. But unfortunately, we “dropped the ball” on communicating why we made the changes.

And what is coming next to help with some of the issues we are currently facing.

Thank you all for your input.

3 Likes

We all hope to make strapi better, Even if we are some complaint users.

I don’t doubt graphql v4 has many improvements, and also has some good aspects for strapi’s future.

Besides that, here is what user want:

  1. if you want keep your graphql roadmap, that’s fine, but “enforce” the users to adopt new way to query is not a smart way.
  2. I do suggest you can have two ways to handle graphql(of course it needs a lot of work). First, enable the user use a normal graphql world query language; second, use your new query to show the task that we can’t complete when we use old graphql query way.

We should also heed the general lesson implicit in the injunction of Ben Franklin in Poor Richard’s Almanack : “If you would persuade, appeal to interest and not to reason.”

I believe tell the user your roadmap, your thinking of graphql are all “reason”, not “interest”.

The user already see the “bad” part of it, can we see the “interest” part of it ? Do we need to endure the bad part to accept the “interest” part?

In terms of engineering, keep the response as same format as restful v4 api is fine, we can easily flat the response to adopt, but the query part is hard to handle(need to clear and easy to read).

2 Likes

This was absolutely our mistake and one that we have certainly clarified that our communication was just straight up really bad before v4 launched. We should have talked more with our users (community and paid) with a two way communication flow to show the interest part of GraphQL and the new structure but also other parts of v4 like the REST population.

It’s something I personally have stressed a lot with our product, engineering, and marketing teams and that’s a big reason why @Paul_Brats is also focusing on it from the devrel side. Some information is absolutely coming to explain more on the interest side and we are making changes at the product level to be way more transparent about things before we build them such as what our expansion squad did with custom fields and their RFCs:

Both of these were to gather user feedback long before we actually started building custom fields (we just started the implementation this quarter for them). And this is what we should have done for many v4 parts including GraphQL.

I just completed a (really awful) migration from v3 to v4
[…]
this migration process was painful and costly in time.

May I ask how complex the CMS data structure was and how long it took you (roughly) to migrate?
My team of two devs is currently trying to migrate as well and at this point I’m not even sure if we can succeed at all.

If this is too off topic, maybe drop me a PM if you’d like :slight_smile:

From further up in the post - my code went from readable to unnecessarily complex (with no benefit).
It becomes very difficult to debug and maintain without doing something to flatten the response to the ‘old strapi 3’ format.

1 Like

Did you try to add interceptors to the response for removing unrequired and nested keys?

As a stopgap I put something on the client - which I know was bad.

I’d love it if there was a plugin to do it - or something already built. I know earlier in the thread there is a bit of code. Anything better out there?

tx

1 Like

I know this might not be as helpful to all of you that largely prefer GraphQL but a community member earlier this year did create a javascript SDK that might help you quite a bit: https://strapi-sdk-js.netlify.app/

I believe sometime in the near future we plan to adopt this and expand it quite a bit and possibly including some simplification of the response through the SDK itself. This provides solutions to both sides as we can build the features with the complexity required but the client can keep the response simple. At least that’s the idea we have been tossing around recently. (This SDK does use REST)

1 Like

Why should we use this? I mean, it is almost like using Axios with an extra layer, am I wrong?

Yes there is an abstraction layer for sure, the end goal being better developer experience on the client side (one the principles that GraphQL was built on too, though it’s debatable on that principle as well considering how subjective that opinion is).

At the end of the day what makes good UX largely boils down to abstraction. You apply a human readable abstraction onto a complex set of code that isn’t human readable. The SDK is as it’s name implies, it’s a kit or set of tools that simply the development of software (AKA a software development kit).

Ideally the purpose of this SDK is to make requesting data and transforming the response easier so you can think less about it on the client side and let the SDK do the heavy lifting.


If you would rather do that manually yourself, then certainly that’s not a problem. Only trying to offer potential solutions for now.

Our idea is to keep it less attached to some backend as possible, following Clean Architecture approaching as long as possible.

This SDK makes the Frontend very attached to Strapi.

Hi my 5 Cents here that with that structure:

{
    "data": [
        {
            "id": 1,
            "attributes": { ... }
        }
    ],
    "meta": { ... }
    }
}

You would absolutely love to have payload type field, then this structure would make much more sense…

{
    "data": [
        {
            "id": 1,
            "attributes": { ... },
            "_type": "image",
        }
    ],
    "meta": { ... }
    }
}
1 Like

Going with GraphQL instead of REST is not only about the transport layer or how the requests and responses are sent.
It also imposes design constraints on the backend, i.e. how one writes the controllers and services.
When you go with GraphQL, you think in terms of resolvers.
When composing a complex response object, each field that it can return can be resolved by a separate resolver that GraphQL plugin will call when the field needs to be populated. This is where the flexibility of GraphQL comes from.
So, the transition from GraphQL to REST based SDK won’t happen for those who already deep in this sh*t (excuse my French here :))
Also, the client side tooling, if you use GraphQL, chances are you are using Apollo client or similar with all their caching and hooks with reactive updates to queries, etc…
Unless SDK can be just plugged in to Apollo link chain just like a transport layer transformer, this SDK won’t see much traction with the part of the community that has been affected by this GraphQL rewrite the most.

These are my 2 cents (and all above is my opinion only).

2 Likes

With the excessive nested data and bloated responses you made the developer experience horrible.
I don’t think any feature or plugin you might add in the future will make up for this.

You most likely forgot the reason why people started using strapi in the first place.

Looking forward to the next major version.

Reading this thread made me really sad.
I see that I need to rewrite and test all of my code in order to make my system work on v4. My system uses both REST and graphql interfaces and from the thread, I see that the structure of the response has changed for both of them.

I understand that you need to change stuff when developing Srapi and I need to change my code to benefit from the new functionality in new versions.
But I do not see any benefits from upgrading to v4. I just see a lot of rewriting and testing that I have to do just to get my code working on v4. I would rather spend time developing my system, than making the system work with v4.

My system sync/read data from 7 external APIs in order to populate a catalog of 100.000 companies. I’m not sure how much time I need to spend rewriting my system. But I know it will not be a joyful task. And when I finish it will be the same as today, with no new functionality.

I have enjoyed using Strapi for the past two years and recommended it to others. But now I have to evaluate if I want to rewrite my code for v4 or find an alternate to Strapi.

I have a lot of rewriting to do either way. So, Strapi, can you please tell me why I should decide to rewrite for v4?

5 Likes

Tried to start a new project yesterday. First time with v4. Then I saw the super strange json structure the REST Api was giving me. I’m consuming the API with native iOS and Android apps.
Especially in iOS with v3 I could ditch DTO objects and could map JSON responses directly to my model structures. I loved this simplicity.

Now I need a workaround, since I’m not willing to adopt this new structure in my apps. Especially with deeper nested objects, it’s a nightmare.

My options are now:

  1. Use Strapi v3.
  2. Use v4 with the transform script @roelbeerens posted above (Discussion regarding the complex response structure for REST & GraphQL (Developer Experience) - #9 by roelbeerens)

What do you guys think is best in general?

@DMehaffy + team. I understand your decisions. Sure, there is a standard you adopt with this structure. It’s good for upcoming features, okay. But at least give us a big switch, like for enabling the publish system. Because those new features are not here yet and even if they’re there, only a subset of users will need it.

Maybe an option would be to extract the transform code of roel in to an officially maintained and tested plugin?

1 Like

We aren’t again making changes but for most of that to happen it will need to wait for v5. We have, for now, committed to this decision for the v4 scope but that is of course subject to change with v5.

Once we get closer to working on v5 we will be opening lines of communication to detail some plans and get feedback wayyyyyyyy in advance unlike the v3 to v4 process.

For now we can’t really spare the engineers to work on a plugin/modification like this but we are happy to assist any community member picking up such a complex topic. We do plan to work on an official javascript SDK in the next 6 months that hopefully will handle most users use-cases (think like the transformer but on the client side instead).

I don’t have exact ETAs on v5 nor the SDK only that I know they are on our internal roadmap.

2 Likes