Save relation items order (Issue #2166)

Thank you for verifying solution option

After playing a round with the problem here is some thoughts about it (for documenting purposes)

I coudnā€™t found a solution from update lfecycle hooks. I implemented a solution 1st with before and after update hooks but failed to serve places with right order. To reproduce the problem manually goes as follows: Create journeys and places as mention before in this thread. Then delete places completely which removes all associations also. Add places in another order in journey. Graphql API return places in some order as before (=always the same order) dispite the order changes in MongoDB record and Rest API.

Next thought was to implement solution with afterFindOne lifecycle hook. I added dummy order attribute the place and in afterFindOne hook loop through the places and set the right order number to each one. Again REST API respected the mutation and show the right order values with the response output. Grapql API failed to do so. It seems that Graphql API response does not go through afterFindOne hook before returning the result.

Dirty solution for now. We are now using order attribute set by content authors to round the problem before v4 landing. In Graphql query we are using sort inside of the main query to sort places. This is not a great solution and will cause some problems while authoring content b/c places can be part of many different journeys.

query JourneyExample {
  journey(id: 1) {
    name
    places(sort: "order:asc, createdAt:asc") {
      name
    }
  }
}

I did just speak to the team earlier today about this and itā€™s not likely we will include relational orders in the v4, at least not initially. There is just too much database overhead to handle something like that and still maintain good performance.

Hi

Since I have no deaper understanding about the issue can you verify that issue is really relational order in v4? Asking this because REST API is fully capable of returning places in right order and all the hooks are also respecting the right order. This is why I wonder if the problem would lie only in GraphQL plugins which is only part of the chain that is loosing it?

Best regards,

I am surprised this is still the issue and has been pushed out of the queue.

Imagine having to tell your customer that in order to properly order their list of items they now have to create a new list somewhere else or assign a order number to ensure they are in correct order even tho UI clearly gives you an option to order manually.

Looks very bad for us(developers) delivering such products and leaves a bad after taste for such an amazing tool you guys have created :slightly_frowning_face:

Donā€™t mean anything bad with this but I am just surprised itā€™s been pushed out of the scope for so long.

2 Likes

@tsvecak itā€™s not a simple thing to implement from a developers view, non-technical users see it as something simple to implement but there is a massive overhead (and grows quickly as the number of related entries also increase).

Itā€™s also extremely rare for most API-based application to allow for a manual ordering like that typically you would just sort on an existing field (usually an ID or a date/time). While we wonā€™t allow for manual ordering in v4 we will offer the ability to sort relations by specific fields.

A simple implementation may be adding a "position" field (or "strapi_position") under the hood for each ContentType entry. And have a drag and reorder functionality for the entries implemented into the UI View below such that when a reorder occurs, the "position" field of the entries under the hood changes to reflect the new order and it syncs with the server database. This new order becomes the default order that strapi feeds out for a standard API call or (for backwards compatibility) the new ordering can be gotten with something like "...contentType/?_sort=position:ASC".

On whether this is a necessary feature, I think itā€™s a big plus. The order strapi feeds out entries of a contentType is in most cases the same order it is naturally implemented in the UI by a developer. I have to say the current order it feeds out entries in itā€™s api responses is plain bad. Today, I had to enforce add a ?_sort=id:DESC on all api requests for a particular project. I had just uploaded new book entries for the project and on the UI expected the new books to show on top of the list but strapi consistently sent old entries as the first 2 entries in the api calls. Imagine pushing a new commit to github and in the commit history UI you see a commit from 2 years ago at the very top. WHAT THE ā€¦!! Thatā€™s very unnatural. I donā€™t even understand the order behaviour as itā€™s not predictable or consistent.

While ordering based on time/order of upload can be enforced with ?_sort=id:DESC as I said earlier, for some use cases like deciding priority for different entries (on the UI view at least, higher up or bottom down), having this is a big blessing. Plus thereā€™s simply no maintainable way to implement this at scale for a high volume of entries. Implementing this is very consitent with the simplicity of strapi which is why I personally love it (and I believe youā€™ll agree too).

Please consider it. Thanks.

2 Likes

I understand itā€™s not that easy, but weā€™ve seen community propose solutions one of which actually does the job(Save relation items order (Issue #2166) - #6 by roelbeerens - clear all the rows and re-save them after the post is saved).
Granted itā€™s a hacky way of doing things but still better than not having an option at all.

After all this is a CMS that weā€™d like to give our customers to work with and not developers to manage.
Only other solution is to get even more hacky with custom field that holds the order or something else that I can order the list by.

No customer out there will be happy with the fact that they canā€™t do a simple action of changing the order of items. Wordpress has this for a long time and all other CMS have it as well.

While Strapi is API based and they usually donā€™t allow manual ordering, strapi advertises itself as a leading open-source headless CMS and this is a basic CMS functionality.

For now I can live with a hack solution that was suggested above but canā€™t see any larger customer being happy with me handing them CMS without these basic functionalities.

1 Like

Iā€™m curious to know what other CMSes youā€™ve noticed this on as we regularly check numerous other CMSes and have noticed that missing feature as well. The primary reason for not having it is a technical and performance challenge on how to properly manage it without causing major performance problems especially in larger applications.

Wordpress has this feature, I dont want to start complaining about something that im using for free but if you want this to get adopted then i am not understanding why the focus seems to be on ā€˜nice to haveā€™ features instead of absolute essentials like this.

How can you expect a user to manage and order possibly thousands of posts by manually setting an ā€˜orderā€™ property?

You should at minimum remove the drag and drop interface because people will assume its there to order things and not for no reason at all.

I mean simplest example is wordpress. Granted itā€™s through usage of Advanced Fields but I doubt thereā€™s many wordpressā€™s out there without advanced fields that do some interesting things. Aside from advanced fields plugin itself, there are other plugins that are pretty much standard in wordpress for re-ordering posts, custom fields etc.

I can see how this can temper with the performance in large applications(talking about thousands items) but I am sure you can see the big issue in having to edit 100s of entries to change the order of items regardless of which approach you take to tackle this.
Otherwise how can our customers manage the order themselves that is not dependant on date or some value? Is there a better alternative?

Reason why I am also surprised this is not considered a bit more is especially as UI offers drag and drop and actually saves that order that youā€™ve created in the UI. Could we somehow gain access to that saved order list?

As dtah mentioned above I donā€™t like complaining about free stuff either but I am pretty sure most of the community would be ready to pay for strapi as well since it provides a lot of benefits, however these benefits are now solely focused on developers rather than final consumers(customers using CMS).

We are already planning to remove that, it was never intended to be on that particular interface and is used elsewhere but accidentally got injected there too.

Since we are not a monolithic CMS but instead a headless one, typically you wouldnā€™t and instead you would sort during the API request time based on the order you need (created/updated/published datetime or some other field).

Iā€™ve never seen a REST or GraphQL API interface that generally allows you to manually order something though I understand your point.

And itā€™s for this exact reason why even injecting automation is extremely damaging to performance, you have to look at this from the technical complexity required to handle something like this at the most base level => the database.

Take for example you have an entry with say 100 relations, then 200, maybe 1000 and you want to manually order those through the interface.

In SQL (Iā€™m throwing out NoSQL since we are dropping support for it with v4) keep in mind you cannot reorder the entries in the database, SQL databases organize entries to the table based on the time they were added and are assigned a auto-increment ID. You canā€™t fundamentally swap IDs of entries (you can but itā€™s not a fun process, and one you certainly wonā€™t be automating) nor should you since itā€™s designed to be the unique ID for that entry. Meaning you now need to add an integer field or something like it to keep track of the order, we already do this with components since you can manually order those.

Now say in the 1,000 relation entry you want to move item 874 to position 4, thatā€™s 870 entries that have to have their order adjusted and it needs to happen fast. That is one very massive bulk update, especially if you arenā€™t scaling the backend (horizontally or vertically). Since node is single threaded and you most likely canā€™t do that big of a query in one go (there are limits to the size of a SQL update query you can make, default on MySQL for example is 1 Mib, you can confirm your maximum with SHOW VARIABLES LIKE 'max_allowed_packet';).

TLDR: Itā€™s easier said then done to actually implement something like this, and the question becomes is it really worth it or is it just a matter of designing a content structure that doesnā€™t rely on manual ordering?

For now you are completely correct, our current focus is on developers. Given we will expand into content creators/managers, ect later on; our core focus right now is still developers.

FYI, Iā€™m not saying this wouldnā€™t be a good feature to add, I am only looking at this from the perspective of a previous community member (Iā€™ve been around since early 2017 and only joined the Strapi team in an official capacity just over a year ago).

In the 4 years that Iā€™ve been around Strapi and all of the other platforms Iā€™ve looked at (Wordpress included) I have never seen a viable reason to manually order content. I also use Strapi myself on various personal projects one of which being a 3rd party API for a video game called Elite Dangerous:

And at the same time itā€™s powering various tooling that Iā€™ve built or other tools in the ED community:

In most, if not all, of these examples time based results are the most common or if itā€™s time based then there are custom filters, sorting, paging, ect depending on the requirement.

1 Like

I have to say, this is tough to hear and really makes me regret choosing Strapi as a CMS. Iā€™ve increasingly been thinking I made a real mistake in looking at this as something that could support even a standard agency website with a small editorial team managing it, and this comment confirms it.

For what itā€™s worth (and this is just my opinion) a CMS has to be a tool for content creators first. My clients and stakeholders really arenā€™t invested in what the developer experience is like, they want something that is going to give them the superpower of managing their site in an intuitive way with little outside assistance.

Now that I know the editorial experience isnā€™t a priority for the team, Iā€™m in the position of having to explain this to my stakeholders and decide whether itā€™s worth it to continue building with this tool, or to cut my losses and rebuild with something tried and true ā€” a very unfortunate scenario.

1 Like

Keep in mind we arenā€™t ā€œjust a CMSā€ but Strapi is a Headless CMS, the intention has always been that some developer experience is required and expected. We are moving more towards that Content manager/editor experience but we need to make sure we get all of the kinks worked out for the developers so they can help build the plugins that everyone needs to have a fantastic experience.

Our focus since June 2021 right now is entirely rebuilding large parts of the project with our v4 (see: Strapi v4 stable is live) which includes an entirely new design system, plugin API, and completely rewriting the backend Database/Query layer from scratch. Our hands are a little full at the moment with such a small team :wink:

As with everything, it takes time. The next 2 big features on our map certain are more geared towards content managers/editors (content versioning and publication workflows) and I would advise you vote for or submit feature requests on https://portal.productboard.com/strapi/1-roadmap/tabs/2-under-consideration as we base a lot our decisions on the data we get there.

Iā€™d like to push back on this and say that almost every website Iā€™ve made that has a marketing strategy attached to it leverages features like this. Hereā€™s a use case Iā€™m trying to accommodate as we speak:

I have a website with a series of case studies outlining the different types of projects a company has worked on over the years. On the homepage, thereā€™s a section where a subset of those case studies are ā€œfeaturedā€ in a carousel/slider.

These features are changed up every once in a while:

  • when projects are updated and they want to promote the changes
  • when the marketing team decides they want to highlight certain types of projects for a period of time
  • when new things launch

The order matters to the marketing team, because part of their strategy involves controlling the visibility of these things.

Maybe this isnā€™t what Strapi was meant to be used for, but thatā€™s not the impression I got when doing a surface-level analysis of it (which is admittedly on me).

2 Likes

This can easily be solved by adding 2 date fields for start_promo_date and an end_promo_date then simply having the frontend filter itā€™s current date to fetch those promo projects.

Or alternatively you can simply add a boolean to set promo status and then use a sort parameter during the request to get those projects first (or in combo with the dates)

There are various ways to handle it currently that arenā€™t intrusive to a content manager, we ourselves do the same thing on the https://strapi.io website. Especially with: Blog Strapi | Open-source Node.js headless CMS (and all itā€™s various tabs and the content loaded on them).

Itā€™s just a matter of building a content model structure that fits what you want to do in the frontend without adding too much complexity that makes a content managers job harder or too complex to easily understand.

Honestly on 2nd thought, a single date field would work perfectly and you could even schedule the promo project swap out wayyyy in advanced and simply filter out dates higher than the current and then sort by that date.

Doing so would be crazy easy and even provide the benefit of being able to make changes in advance and your frontend could automatically update on a schedule.

(I only thought of this, as I was looking at the Strapi project that runs our website, and itā€™s literally exactly what we do with blog posts :laughing: )

Yeah, I get what youā€™re saying (and appreciate the tips).

There are places that we do similar things on the site, but letā€™s say Iā€™ve got a well of 40 projects to choose from, and I want to highlight 4 of them by setting the same start_promo_date and end_promo_date values for each. In that case Iā€™d need another field (something like a weight or order field) if I also wanted to control the order that they appear in arbitrarily.

If thatā€™s right, it just seems like a more cumbersome ask for a content editor (open each project entry individually, pick the dates you want, and then add an order number to them, and remember to change all those things when you update new promos) vs just having a draggable list with a nice typeahead interface on the homepage model.