Discussion regarding the complex response structure for REST & GraphQL (Developer Experience)

Discussions
1

This topic is not about population, please discuss that topic here: Discussion regarding default population of REST

Current Status Post: Discussion regarding the complex response structure for REST & GraphQL (Developer Experience) - #35

Hello Everyone,

As mentioned in a news post in our Discord last week we wanted to setup this discussion around how the response structure is handled (nested data & populate) in Strapi v4.

If you haven’t already please read the following documents, pull requests, or issues before commenting:

Currently purposed suggestions:

  • Revert to v3 style in GraphQL => Not possible at this moment
  • Remove Attributes and throw everything under data in GraphQL => Not possible due to breaking changes
  • Recommend usage of something like normalizr for REST Users + potentially integrate a normalization library in the Strapi SDK JS

Current status on v4:

At this time we are not prepared or planning to modify this data structure in the response, the time for that type of modification has passed as it should have been addressed during the RFC process. We do understand that it was not clear at the time during our RFC process this was the case and we are already doing retrospectives to improve this process.

In the short term we would rather explore options to optimize using this response structure on the client side and see how we can assist. An above mentioned suggestion of including normalization libraries in best practices or even sponsored SDKs (like the Strapi-SDK-JS is) is one example of what we believe could benefit everyone.

Why is Strapi using this structure?

The primary goal behind this new structure was to pick some kind of standard for a response structure, ideally not reinvent the wheel and try to use an already existing standard. In our case we picked JSON API: https://jsonapi.org/

image
(Image src: xkcd: Standards)

We have not entirely implemented the standard yet but our goal with the first release of Strapi v4 was to lay the groundwork so that we could continue to build on it without breaking changes. This meant unifying the REST and GraphQL data responses, error handling, filtering/paging system, ect.

2 Likes
3

Thank you very much for opening a space for a discussion. I think it should be clear that you had the best intentions with the change.

However, the GraphQL response in v4 feels very complicated and cumbersome. I understand there shouldn’t be a breaking change and I’m happy you follow the semver conventions.

What about a non-breaking change, which still reverts part of it: you could output the data under attributes and the root object at the same time. Instead of changing the schema, it would extend it and therefore be a non-breaking change:

query posts {

    // current v4
    data {
      attributes {
          title
      }
    }

    meta {}

    // desired v4 alternative
    data {
      title
    }
}

At some point the question was raised where to put the meta information in this flat case and somebody mentioned it could just be repeated with every entity:

query posts {
    data {
      title
    }

    meta {}
}

would lead to:

[
  data: {
    {
        title: ''
    }
  },

  meta: {}
]

I don’t know what the implications would look like from an implementation perspective, but from a users perspective this wouldn’t be bad.

Is this something worth considering?

1 Like
4

This is actually intentional, there are multiple uses for meta. You have the meta for parent content-type you are currently requesting and each level down there is potential for additional meta.

The parent meta is for things like pagination and sub-meta could include things like i18n and in the future versions of content, ect.

5

@DMehaffy I just realized I made a mistake. Take this example from the Github GraphQL API, which I’d say shows the desired behavoir:

{
  repositories {
    nodes {
      id
      name
      issues {
        nodes {
          author {
            login
            ... on Organization {
              id
              name
            }
          }
        }
      }
    }

    pageInfo {
      hasNextPage
    }
  }
}

To re-build that in strapi you’d have to create the following query:

{
  repositories {
    data {
      id
      attributes {
        name
        issues {
          data {
            attributes {
              author {
                data {
                  attributes {
                    login
                    ... on Organization {
                      data 
                        id
                        attributes {
                          name
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }

    meta {
        pagination {
          pageCount
        }
    }
  }
}

As you can see the strapi response is much deeper, because attributes is always nested under data. I guess the desired behavoir would be that everything that right now lives in attributes would be available under data (more common is nodes though, because GraphQL is about nodes, but let’s ignore that for now) too (to make this a non breaking change).

{
  repositories {
    data {
      id
      name
      issues {
        data {
          author {
            data {
              login
                ... on Organization {
                data 
                  id
                  name
                }
              }
            }
          }
        }
      }
    }

    meta {
        pagination {
          pageCount
        }
    }
  }
}

This would make the situation already much better.

5 Likes
6

Personally I don’t really get the issue of the data response being to ‘deep’.

I do however mainly use REST and the new structure of the API in my opinion is much more structured and more consistent. This gives me the opportunity to write resolvers that can be used for multiple entities. It also makes data response more predictable.

Looking at the usage for GraphQL, yeah, it feels a little bit tedious to write data > attributes over and over again, but thats just the structure of the response/Strapi. You just need to write it once anyway. You could also opt for using fragments for your entities (highly recommended anyway).

In terms of using your data and finding your data set in your response, I’d go for a function in a frontend framework where I’d put my response in and it returns my data from the attributes object.

2 Likes
7

@roelbeerens My main issue with that isn’t the query itself and I think the data transformation you’ve mentioned is totally possible. My issue with that is, that I’d like to co-locate queries and components (in fragments). Ideally I’d like the component to receive the data through props.

  1. If I add a transformation step in between, to reduce the depth, the GraphQL Fragment doesn’t match the data received through props anymore.

  2. If I receive the deep object, this becomes a pain to work with in the component itself, plus the JSON that next.js etc. are passing down becomes huge and therefore hydration more expensive.

2 Likes
8

I see what you’re getting at. I understand in your case it would be painful.

You do have a solid argument in regards to hydration. I haven’t thought of that for now, but it could have a serious impact on the Frontend.

It’s hard to think of a solution when the structure will be the same.

I do not know if the GraphQL should have the same response as REST in terms of consistency but maybe it’s a solution to just change the response for the GraphQL plugin?

1 Like
9

When working with the new response for a few days, it somehow started to bug me to mistype attributes every single time and missing the ‘old’ response. I decided to write a normalizer function which looks into deep objects & arrays. In case somebody want’s to work with this, here’s the code. It’s not implemented into the API, but into Nuxt in the frontend.

const normalize = (data) => {
  const isObject = (data) =>
    Object.prototype.toString.call(data) === '[object Object]'
  const isArray = (data) =>
    Object.prototype.toString.call(data) === '[object Array]'

  const flatten = (data) => {
    if (!data.attributes) return data

    return {
      id: data.id,
      ...data.attributes
    }
  }

  if (isArray(data)) {
    return data.map((item) => normalize(item))
  }

  if (isObject(data)) {
    if (isArray(data.data)) {
      data = [...data.data]
    } else if (isObject(data.data)) {
      data = flatten({ ...data.data })
    } else if (data.data === null) {
      data = null
    } else {
      data = flatten(data)
    }

    for (const key in data) {
      data[key] = normalize(data[key])
    }

    return data
  }

  return data
}
17 Likes
10

I think that this new structure makes everything much more complex.

With this level of nesting…

I would love to see this structure simplified.

Also this is a plugin, I think a new v5 version could even be created with breaking changes while Strapi stays at v4.

3 Likes
11

This was super helpful, thanks for sharing it!

12

+1 on the complexity of the structure. Really dislike it and hope there is a solution such as the one @gustavpursche or @abdonrd suggested

I’ve attached a picture of a pretty gross before and after converting one of my queries from v3 to v4. This is not anything unusual. I would say most of my queries have become this uglified

Screen Shot 2022-01-03 at 1.25.44 PM
Screen Shot 2022-01-03 at 1.25.44 PM2454×2110 381 KB

7 Likes
13

I have built a function in addition:

export default async function get<T>(collection: string, fields?: string[], one?: boolean) {
    const {findOne} = useStrapi4()
    const {find} = useStrapi4()

    return normalize(one
        ? await findOne<Strapi4Response<T>>(collection, 1, {
                populate: '*',
                fields: fields
            }
        ) : await find<Strapi4Response<T>>(collection, {
                populate: '*',
                fields: fields
            }
        )
    )
}

<script setup lang="ts">
import get, {Description} from '~/utils/useStrapi'

const response = await get<Description>('descriptions', ['header'], true)
</script>
14

Mind sharing how this is used - am on Nuxt as well (with just barely enough tech knowledge to be dangerous I guess)

15

I have another idea regarding a possible solution: you could release a v5 version of the GraphQL Plugin and either create feature parity with the v4 version of it while strapi is on v4 or fade-out the v4 version of the plugin and only maintain v5. That way it would be possible to release a version which could fix this without making it a breaking change …

1 Like
16

All of our packages keep the same version on release, we do this intentionally as it eases our release process from a mono-repo and makes the process of upgrading between versions very clear.

It’s unlikely the engineering team would agree to release a v5 of GraphQL before all of the other packages and it’s far too early to start working on a v5 for anything else.

17

@DMehaffy thanks for feedback, can you guys do something about this new Data attribute fields? Not sure what problem it was trying to solve. It’s only making things more complicated. More and more people are going to use the Graphql plugin as it’s becoming standard now.

1 Like
18

Unsure yet, I started these discussion topics (this one and population) as a means of gathering quick feedback so that we had more context to discuss internally on options to help users.

For now we have not had those discussions but are passing ideas around here and there. The feedback has been heard though no question.

When I have more to share I will :slight_smile:

1 Like
19

As requested by @DMehaffy , I’m copy-pasting my github issue related to this discussion here:

-----------------------------------------------------------8<-----------------------------------------------------------

Bug report

Describe the bug

Because of id and attributes split in API schema the GraphQL cache can’t be updated and raise a warning in the browser console (among obvious cache handling issue).

The warning message from the console is:

invariant.ts?6cf4:42 Cache data may be lost when replacing the attributes field of a ExampleEntity object.

To address this problem (which is not a bug in Apollo Client), either ensure all objects of type Example have an ID or a custom merge function, or define a custom merge function for the ExampleEntity.attributes field, so InMemoryCache can safely merge these objects:

  existing: {"__typename":"Example","createdAt":"2021-12-25T23:43:56.648Z","firstName":"John","lastName":"Do","message":"I'm asleep","status":"NEW","dueDate":null,"referer":"https://example.com","contactDetails":[],"user":{"__typename":"UsersPermissionsUserEntityResponse","data":null}}
  incoming: {"__typename":"Example","dueDate":null,"firstName":"John","lastName":"Do","message":"I'm asleep","referer":"https://example.com","ipAddress":"12.34.56.78","createdAt":"2021-12-25T23:43:56.648Z","updatedAt":"2021-12-25T23:43:56.648Z","contactDetails":[],"communications":[]}

For more information about these options, please refer to the documentation:

  * Ensuring entity objects have IDs: https://go.apollo.dev/c/generating-unique-identifiers
  * Defining custom merge functions: https://go.apollo.dev/c/merging-non-normalized-objects

This is because the GraphQL schema is the following:

  • example query:
example(id: ID): ExampleEntityResponse
type ExampleEntityResponse {
  data: ExampleEntity
}
id: ID

and

data: ExampleEntity
type ExampleEntity {
  id: ID
  attributes: Example
}

Steps to reproduce the behavior

  1. Using GraphQL API, request the same asset in a query and in the result of a mutation (for example)
  2. See warning message in the browser console

Expected behavior

The id and attributes split in the GraphQL API schema is incomprehensible for me. I can’t understand why there are not in the same object (like Apollo Client is obviously expecting to find it). In addition, I also can’t understand why we need to wrap everything in a data object while fetching a single asset. I guess there is strong reasons for such things, but I can’t actually believe them :exploding_head:

IMHO, the expected schema should be like this:

example(id: ID!): Example
type Example {
  id: ID!
  createdAt: DateTime!
  updatedAt: DateTime!
  // other fields...
}

Screenshots

N/A

Code snippets

Regardless the GraphQL cache issue, I’m currently forced to write such code in every query/mutation (and I really hate it :angry:):

function Example({ id, attributes }) {
  Object.assign(this, { id, ...attributes });
}

this.$apollo
  .query({ query: examplesQuery, variables: { ids: this.ids } })
  .then(({ data: { examples: { data } } }) => {
    const examples = data.map(e => new Example(e));
    // finally some useful code
  }

Strapi v3 use by far more “dev friendly” in this respect (no useless data wrapper for single asset, no incomprehensible id and attributes split).

System

  • Node.js version: v16.13.1
  • NPM version: v8.1.2
  • Strapi version: v4.0.5
  • Database: MariaDB
  • Operating system: Archlinux
    -----------------------------------------------------------8<-----------------------------------------------------------
6 Likes
20

I have nested objects as well and have ported over to the new graphql format already but now am adapting the front end code and it is unreadable, prone to errors, really hard to debug.

Not very usable without a workaround, fix or plugin

For my use case:
Strapi 3: this.companies[i].investments[0].fund.id
Strapi 4: this.companies.data[i].attributes.investments.data[i].attributes.fund.data.id

Authoring the GraphQL queries isn’t too bad but using them on the front end is tough. If this were the approach in Strapi 3 I’d have gone with something else.

I’m happy to help w/ feedback, test etc. rather than just complain. Know you are all balancing design/usability etc.

2 Likes