Discussion regarding default population of REST

roelbeerens · December 14, 2021, 10:58am

Thanks for opening a topic about this!

I think you guys did a good thing to NOT auto populate everything by default. When I first started using Strapi I found it pretty neat to have all my data at my disposal. However, after working with Strapi for a while, I found myself constantly editing my controllers and excluding relations in my queries. Why? Mainly for performance reasons. Fetching related data that isn’t used is a waste of resources.

In terms of security, I think it’s a very wise decision to make it ‘secure by design’. More unexpected data in your response means more testing for possible data that doesn’t need to be exposed. Something we have encountered a couple of times, and one of the reasons why we don’t populate entities anymore, unless it is verified the data is secure.

I do however like to have a little bit more control over the usage of the find to be able to populate a relation. It doesn’t feel ‘right’ to open up a controller to populate a relation of an entity. Also, i’d like to have it more consistent (when a relation is one-to-one, I’d expect to open up the findOne controller, and not the find).

DMehaffy · December 14, 2021, 4:57pm

Additional information in this issue: NOTICE: Population of Relations, Media, Component, and Dynamic Zone fields · Issue #11787 · strapi/strapi · GitHub

ovlb · December 14, 2021, 5:03pm

Agreed that not populating by default is the better choice.

In the GitHub issue, there was another part of this discussion though, the question of the API key permissions. Currently populating is not working with read-only keys. But giving an API key full access to Strapi to populate the data even though you’d not expect this key to ever write or update data does not feel too good. Is there another discussion open regarding this?

DMehaffy · December 14, 2021, 6:31pm

Can you explain a bit more as I’m not quite understanding what you mean around the API key.

Population permissions don’t really have too much to do with what auth you are using (users-permissions or API key) other than with users-permissions you have to have the find option enabled related collection type

ovlb · December 14, 2021, 7:31pm

Sure.

So, I wrote about problems I had with populating, and another user came up with this solution.
As long as the type of my API key is set to «Read-Only» the relations will never show up, regardless of how I format my request (e.g. populate=* or populate[0]=authors&populate[1]=categories). Also this solution for deeply populating objects does not work. The only solution currently seems to be changing the token type to «Full access» .

DMehaffy · December 14, 2021, 9:05pm

This is interesting and shouldn’t be happening. This particular issues needs a dedicated bug report
Can you open one on GitHub for this?

ovlb · December 15, 2021, 8:59am

Yes, will do so later today.

Meher_Chandan · December 15, 2021, 7:51pm

is there any way to populateimages using query engine when images are available under component within component lying under dynamic zone.
Attribute in Schema.json

"attributes": {
    "section": {
      "type": "dynamiczone",
      "components": [
        "clps-components.banner",
        "clps-components.carousel",
        "clps-components.promotional-banner-with-products",
       ...
      ],
      "pluginOptions": {
        "i18n": {
          "localized": true
        }
      },
      "required": true
    },
    "category": {
      "type": "relation",
      "relation": "oneToOne",
      "target": "api::sub.sub-categories"
    },

I am trying to populate all the nested images in components in dynamic zone using query engine api,

await strapi.db.query("api::xlps.xlps").findOne({
        where: { locale: language, category: { id: category.id } },
        populate: ["section", "category"],
      });

Here everything is being populated except the media files. Can you suggest how it can be done without using Rest API. Also its not possible to name all the image field names as I have around 30 components under dynamic zone

DMehaffy · December 15, 2021, 8:11pm

Is the structure of that DZ => Compo => Compo => Media or DZ => Compo => Media

And can you give me just one example and I can write a sample query to show you.

Meher_Chandan · December 16, 2021, 5:21am

Thanks @DMehaffy. I have both structure. Here is a sample schema for component under DZ

"attributes": {
 "image": {
      "type": "media",
      "multiple": false,
      "required": false,
      "allowedTypes": [
        "images"
      ]
   
    },
    "productCuratedFilterBoxSmall": {
      "type": "component",
      "repeatable": true,
      "component": "clps-components.product-curated-filter-box-small",
      "max": 2
    },
}

Here DZ->component has the image field and there is another image field under product-curated-filter-box-small.

DMehaffy · December 16, 2021, 5:57am

So using qs is the best method to build these types of filters. Below is an example I am running in a replit.com to quickly get the stringfied query string:

const qs = require('qs');
const query = qs.stringify({
  populate: [
    'section.image',
    'section.productCuratedFilterBoxSmall.image',
    //ect
  ]
}, {
    encodeValuesOnly: true,
  });

console.log('\n' + query)

which dumps out: populate[0]=section.image&populate[1]=section.productCuratedFilterBoxSmall.image

This is a basic population, there are more advanced ways to build these (which we are working on new docs for). The odd thing with dynamic zones is you can’t specify the specific component you want to populate in the DZ so the population looks more like:

DZ => ~~compo~~ => nested compo/field => ect

It’s quite odd but I’ve confirmed with @alexandrebodin this is intended for DZs

Meher_Chandan · December 16, 2021, 6:00am

Thanks for the response @DMehaffy. I’ll try the solution.

DMehaffy · December 16, 2021, 6:05am

This WIP PR documentation that I’m working on has some other examples too: Population for REST API - Strapi Developer Docs

(I’ve more or less rewritten all of our operator docs to provide more complex and complete examples using qs since it’s what we recommend to everyone to help them understand the complex stuff you can do now )

Still working on it and cleaning things up before we merge those docs and deploy them.

Meher_Chandan · December 16, 2021, 8:54am

Thanks @DMehaffy . The documentation is very helpful.

Meher_Chandan · December 16, 2021, 9:25am

Hi @DMehaffy Based on the solution provided, finally I am able to populate all nested images inside the DZ component. Below is the solution I use. It’s working fine for DZ but when I added the same for a relation, its not working. Can you suggest anything

   const query = qs.stringify({
          filters: {
            $and: [
              {
                locale: {
                  $eq: language,
                },
              },
              {
                category: {
                  id: {
                    $eq: category.id,
                  },
                },
              },
            ],
          },
          populate: {
            category: {
              populate: "*",
            },
            section: {
              populate: "*",
            },
          },
        },
        {
          encodeValuesOnly: true,
        }
      );
  const baseURL = `http://${strapi.config.host}:${strapi.config.port}`;
  const { data } = await axios.get(`http://localhost:1338/api/clps-data?${query}`);

Here section is a DZ and category is a relation. Here is the schema

"attributes": {
    "section": {
      "type": "dynamiczone",
      "components": [
        "clps-components.banner",
        "clps-components.carousel",
        "clps-components.promotional-banner-with-products",
       ...
      ],
      "pluginOptions": {
        "i18n": {
          "localized": true
        }
      },
      "required": true
    },
    "category": {
      "type": "relation",
      "relation": "oneToOne",
      "target": "api::sub-categories.sub-categories"
    },

Meher_Chandan · December 19, 2021, 7:46am

Hi @DMehaffy, @alexandrebodin - any suggestion to populate relation in above case?

WardenCommander · December 20, 2021, 7:16pm

Hello all,
is there possibly to populate relation/ component/dynamic zone in lifecycle hook?

example

beforeUpdate: async (data) => {
    console.log("beforeUpdate Lifecycle hook calendar")
    console.log(data);
  },

here data only show data from collection type

Joo · December 21, 2021, 1:28am

I would also appreciate a solution for this, I currently can’t get all my fields since I want to query my strapi api from a React Frontend and can’t use a full access token for that reason.

The alternative solution of setting the find permission for the Public user did not work for me.

ovlb · December 21, 2021, 10:59am

I’ve opened a GitHub issue, where you can add your voice/context.

github.com/strapi/strapi

Relations are not populated if API key is read-only

opened 12:05PM - 15 Dec 21 UTC

ovlb

## Bug report ### Describe the bug In v4.0.0 relations inside API response…s do not show up if the API key is set to «read-only». For this it does not matter how the `populate` query param is formatted (e.g. `populate=*` or `populate[0]=authors&populate[1]=categories`). This [solution for deeply nested relations](https://github.com/strapi/strapi/issues/11836#issuecomment-989131330) does not work either. The only solution so far is to set the type of the API key to «Full Access». While this might be fine in some circumstances, it is not in other. (E.g. I’m currently building an 11ty site with Strapi as a CMS and the 11ty build should only have read access. ### Steps to reproduce the behavior 1. Go to Settings -> 2. Create a new API key with read-only permissions 3. Create a new model w/ some relations (the restaurant example from the docs or [this one](https://github.com/strapi/strapi/issues/11787#issuecomment-986232765)). 4. Request the page in Paw or another API client, using the API token, and the `populate=*` query param 5. Relations are not showing up ### Expected behavior Relations should be populated ### Screenshots Figure 1: API response with a read-only API key <img width="1352" alt="" src="https://user-images.githubusercontent.com/7645967/146183045-89ef16e3-0f21-4e7b-9efe-13b6f781257a.png"> Figure 2: API response with a full access API key (expected behaviour) <img width="1348" alt="" src="https://user-images.githubusercontent.com/7645967/146183202-a8d72a6f-843d-4c45-b886-160555d05a57.png"> ### System - Node.js version: v14.5.0/v16.13.1 - NPM version: v6.14.5/v.8.1.2 - Strapi version: v4.0.0 - Database: Postgres - Operating system: macOS/Ubuntu