How to work with Strapi without loosing data?

System Information
  • Strapi Version: 4.8
  • Operating System: linux
  • Database: mysql
  • Node Version: 18.4
  • NPM Version: 9.5.0
  • Yarn Version: 1.22.19

Hi,
Me and my team started to work with strapi for a middle sized project.
We encountered some issues and I have a question about best practices / advice how to work with strapi on a middle sized team.

So, one of the problems is that if a field is renamed, all the data in that field is gone.
Or even worse, if I rename a component, which has multiple fields, all the data in these fields is gone.
We hope we wont rename anything, but in the beginning we try to refactor stuff if something is not right (eg renaming a singular named component called “subSection” to “subSections” because it is a repeatable component)

Any advice of how we could rename things without loosing data?

Another question regarding strapi transfer feature. It works fine for pushing data to other instances (using the --to param) but it is not working for --from (yet?) I checked the code / error, looks like the feature is not implemented yet. The error Remote Strapi source provider not yet implemented made me think that I should add some provider to the config or something.

Unfortunately we don’t have a direct connection to the database / strapi server (on remote) so we can’t modify the data there or execute some commands (we are using docker images)

One of our tasks is to fill strapi with data. It is a very manual task. The pages we are using (mostly single types) are complex. It would be faster to write some script for this task. Or are there any other things we could try? I see there is this “migration” feature. Could it be used not only for schemas but also for data?

1 Like

Would love to know as well, I struggle with the same issue. I very often change my data model (most of the times it’s just renaming a field), and each time I loose a lot of data

Great question, I want to know the same hopefully someone can answer.

I have not personally used “migrations” feature, but reading the docs, the first line says that it can be used for table structure i.e. schema. Maybe worth a try?

Database migrations exist to run one-time queries against the database, typically to modify the tables structure or the data when upgrading the Strapi application.

No one else? So you can never change your data model? :sweat_smile:

1 Like

Great question, I’d also like to have more information about this!

I can only describe how we work during development; however, we couldn’t find the best / optimal way to roll out / deploy to productions when you use cluster (meaning more than one instance of STRAPI).

During development, we use GitHub (private repository - with two types of Github actions.

  1. for development server
  2. for production server (use as a template to roll out changes)

On strapi side, we installed strapi-plugin-config-sync .

Challenges we are facing and current solutions (not the best)

Issue 01 Versions:

The schema of a major collection needs to be changed

Solution:

Create a new strapi cluster named it vXX and migrate data.
Reason:

  • The main reason: Avoid data loss when applying changes to collections.
  • Allowing front-end developers to test and adopt changes without critical downtime.

Challenges

  • Long process
  • Need to manage multiple servers and databases
  • required sysadmin (this depends on how your infrastructure is maintained)

Issue 02 adding fields:

As long as it does not change the schema

Solution:

We are using strapi-plugin-config-sync & strapi-plugin-import-export-entries more than we are using strapi cli

Challenges

  • All our media files installed externally on CDN we must avoid mismatch of data

Media files are our biggest issue. In our case, some of them are attached to a user, not just images for the website.

Issue 03 media files:

The biggest challenges we are having.

Solution:

  • The first step was to modify/expand the original connector to AWS to support saving in folders
    /ec2-bucket/year/month/[sec/hour/day]/filename see: provider-upload-aws-s3-extended. This solution helped to keep the AWS S3 database short and allow to query and write was script to ready s3 to resync back strapi
  • The rsync is done externally by using batch and only in case of emergency.
  • We handle large amounts of media files, and scanning the S3 folder with 6-figure files takes a long time. Splitting it into folders helped to keep it low-numbered.

It Would be nice to see this feature part of the standard strapi and avoid handling one more plugin.

UNDER TESTING - MEDIA SOLUTIONS:

We are currently testing the following options:

  • Use external solutions for media files instead of strapi.
  • Use a dedicated strapi instance only for media and create a custom field using this server. preferred method

Final word:
the challenges we face are no different from any other large environment we used before strapi.
Databases, media, and schema are ongoing issues. Only a working procedure can solve them.

Let us know if this set of highlighted guidelines helped or if you have different suggestions :slight_smile:

1 Like