Discussion regarding default population of REST

Discussions
1

This topic is not about response structure, please discuss that topic here: Discussion regarding the complex response structure for REST & GraphQL (Developer Experience)

Hello everyone,

As mentioned in a news post in our Discord last week we wanted to setup this discussion around how we are handling population of relations, media fields, dynamic zones, and components in Strapi v4.

If you haven’t already please read the following documents, pull requests, or issues before commenting:

Currently purposed features:

  • Addition of more wildcard filter options (eg: **, *.*, someRelation.*, ect)
  • Option to specify a default population via some file or other method per content-type (default without modification will be no population)
  • Add a global config option to allow populating some number of levels

Current status on v4:

We will not auto populate any field by default within Strapi as the purpose for not doing so is largely performance but also security as within the v4 if a role doesn’t have access to the find controller on a related content-type it is not possible to populate it.

Making this change on v4 is not possible as it would be considered breaking, but we are open to feature requests that extend past the default.

Future status on the future (v5, v6, v7+):

As we look to the future we want to hear from you on how you consume data, why you need deep population of data vs multiple requests. What would make things easier and why.

Likewise we want to start educating users on good practices and hopefully documenting best practices from us on usage of various parts of Strapi (not just population).

1 Like
3

Thanks for opening a topic about this!

I think you guys did a good thing to NOT auto populate everything by default. When I first started using Strapi I found it pretty neat to have all my data at my disposal. However, after working with Strapi for a while, I found myself constantly editing my controllers and excluding relations in my queries. Why? Mainly for performance reasons. Fetching related data that isn’t used is a waste of resources.

In terms of security, I think it’s a very wise decision to make it ‘secure by design’. More unexpected data in your response means more testing for possible data that doesn’t need to be exposed. Something we have encountered a couple of times, and one of the reasons why we don’t populate entities anymore, unless it is verified the data is secure.

I do however like to have a little bit more control over the usage of the find to be able to populate a relation. It doesn’t feel ‘right’ to open up a controller to populate a relation of an entity. Also, i’d like to have it more consistent (when a relation is one-to-one, I’d expect to open up the findOne controller, and not the find).

1 Like
4

Additional information in this issue: NOTICE: Population of Relations, Media, Component, and Dynamic Zone fields · Issue #11787 · strapi/strapi · GitHub

5

Agreed that not populating by default is the better choice.

In the GitHub issue, there was another part of this discussion though, the question of the API key permissions. Currently populating is not working with read-only keys. But giving an API key full access to Strapi to populate the data even though you’d not expect this key to ever write or update data does not feel too good. Is there another discussion open regarding this?

6

Can you explain a bit more as I’m not quite understanding what you mean around the API key.

Population permissions don’t really have too much to do with what auth you are using (users-permissions or API key) other than with users-permissions you have to have the find option enabled related collection type

7

Sure.

So, I wrote about problems I had with populating, and another user came up with this solution.
As long as the type of my API key is set to «Read-Only» the relations will never show up, regardless of how I format my request (e.g. populate=* or populate[0]=authors&populate[1]=categories). Also this solution for deeply populating objects does not work. The only solution currently seems to be changing the token type to «Full access» .

1 Like
8

This is interesting and shouldn’t be happening. This particular issues needs a dedicated bug report :slight_smile:
Can you open one on GitHub for this?

9

Yes, will do so later today.

1 Like
10

is there any way to populateimages using query engine when images are available under component within component lying under dynamic zone.
Attribute in Schema.json

"attributes": {
    "section": {
      "type": "dynamiczone",
      "components": [
        "clps-components.banner",
        "clps-components.carousel",
        "clps-components.promotional-banner-with-products",
       ...
      ],
      "pluginOptions": {
        "i18n": {
          "localized": true
        }
      },
      "required": true
    },
    "category": {
      "type": "relation",
      "relation": "oneToOne",
      "target": "api::sub.sub-categories"
    },

I am trying to populate all the nested images in components in dynamic zone using query engine api,

await strapi.db.query("api::xlps.xlps").findOne({
        where: { locale: language, category: { id: category.id } },
        populate: ["section", "category"],
      });

Here everything is being populated except the media files. Can you suggest how it can be done without using Rest API. Also its not possible to name all the image field names as I have around 30 components under dynamic zone

11

Is the structure of that DZ => Compo => Compo => Media or DZ => Compo => Media

And can you give me just one example and I can write a sample query to show you.

12

Thanks @DMehaffy. I have both structure. Here is a sample schema for component under DZ

"attributes": {
 "image": {
      "type": "media",
      "multiple": false,
      "required": false,
      "allowedTypes": [
        "images"
      ]
   
    },
    "productCuratedFilterBoxSmall": {
      "type": "component",
      "repeatable": true,
      "component": "clps-components.product-curated-filter-box-small",
      "max": 2
    },
}

Here DZ->component has the image field and there is another image field under product-curated-filter-box-small.

13

So using qs is the best method to build these types of filters. Below is an example I am running in a replit.com to quickly get the stringfied query string:

const qs = require('qs');
const query = qs.stringify({
  populate: [
    'section.image',
    'section.productCuratedFilterBoxSmall.image',
    //ect
  ]
}, {
    encodeValuesOnly: true,
  });

console.log('\n' + query)

which dumps out: populate[0]=section.image&populate[1]=section.productCuratedFilterBoxSmall.image

This is a basic population, there are more advanced ways to build these (which we are working on new docs for). The odd thing with dynamic zones is you can’t specify the specific component you want to populate in the DZ so the population looks more like:

DZ => compo => nested compo/field => ect

It’s quite odd but I’ve confirmed with @alexandrebodin this is intended for DZs

1 Like
14

Thanks for the response @DMehaffy. I’ll try the solution.

15

This WIP PR documentation that I’m working on has some other examples too: Population for REST API - Strapi Developer Docs

(I’ve more or less rewritten all of our operator docs to provide more complex and complete examples using qs since it’s what we recommend to everyone to help them understand the complex stuff you can do now :slight_smile: )

Still working on it and cleaning things up before we merge those docs and deploy them.

2 Likes
16

Thanks @DMehaffy . The documentation is very helpful. :slight_smile:

17

Hi @DMehaffy Based on the solution provided, finally I am able to populate all nested images inside the DZ component. Below is the solution I use. It’s working fine for DZ but when I added the same for a relation, its not working. Can you suggest anything

   const query = qs.stringify({
          filters: {
            $and: [
              {
                locale: {
                  $eq: language,
                },
              },
              {
                category: {
                  id: {
                    $eq: category.id,
                  },
                },
              },
            ],
          },
          populate: {
            category: {
              populate: "*",
            },
            section: {
              populate: "*",
            },
          },
        },
        {
          encodeValuesOnly: true,
        }
      );
  const baseURL = `http://${strapi.config.host}:${strapi.config.port}`;
  const { data } = await axios.get(`http://localhost:1338/api/clps-data?${query}`);

Here section is a DZ and category is a relation. Here is the schema

"attributes": {
    "section": {
      "type": "dynamiczone",
      "components": [
        "clps-components.banner",
        "clps-components.carousel",
        "clps-components.promotional-banner-with-products",
       ...
      ],
      "pluginOptions": {
        "i18n": {
          "localized": true
        }
      },
      "required": true
    },
    "category": {
      "type": "relation",
      "relation": "oneToOne",
      "target": "api::sub-categories.sub-categories"
    },
18

Hi @DMehaffy, @alexandrebodin - any suggestion to populate relation in above case?

1 Like
19

Hello all,
is there possibly to populate relation/ component/dynamic zone in lifecycle hook?

example

beforeUpdate: async (data) => {
    console.log("beforeUpdate Lifecycle hook calendar")
    console.log(data);
  },

here data only show data from collection type

1 Like
20

I would also appreciate a solution for this, I currently can’t get all my fields since I want to query my strapi api from a React Frontend and can’t use a full access token for that reason.

The alternative solution of setting the find permission for the Public user did not work for me.