Importing existing data and relations

System Information
  • Strapi Version: 3.4.1 CE
  • Operating System: OS X High Sierra
  • Database: mysql 8
  • Node Version: 14.15.3
  • NPM Version: 6.14.9
  • Yarn Version: 1.22.10

I am migrating a huge (25K articles) editorial website from a hand-rolled php configuration to Strapi. This is much more complex than a blog, my priority is to port as much of the existing content as possible, and to recreate the relations that exist. So I am creating a schema in Strapi that closely mirrors the one used by the current php app.

This is a one-off migration process (future content will be added via the CMS by the editorial team). I have successfully tried to use the plugin by Github user lazurey, but it doesn’t support importing relations. So I am trying to replicate the approach from this Strapi YouTube video, however the author just talks at the end about importing relations, and doesn’t provide sample code.

As a minimal example here, since this is a music webzine, I want to import the following two related tables:

Artists:

  • id
  • name

Records:

  • id
  • title
  • year
  • label
  • genre
  • artist_id

I have this data in JSON and I was able to import it using the plugin save for the relation.

I have two separate requests for help:

1- using the code provided in the video linked above, I can successfully print out the JSON and parse its fields, but when iterating over the JSON I hit a ValidationError with no further explanation. The plugin imports successfully which makes me think I need just a minor update to my code.

./bootstrap.js

 module.exports = () => {
    var input = require('fs').readFileSync('./data/artisti.json')
    const data = JSON.parse(input)
    console.log(data) #prints correctly
    strapi.query('artists').delete()
    console.log(data[0].name) #also prints correctly
    data.forEach(entity => {
        strapi.services.artists.create({
            name: data.name
          });
      });
    };

Error message:

(node:35420) UnhandledPromiseRejectionWarning: Error: ValidationError
at Object.validateEntityCreation (/Users/rxm/Library/Mobile Documents/com~apple~CloudDocs/ondarock/node_modules/strapi/lib/services/entity-validator/index.js:179:25)
at async Object.create (/Users/rxm/Library/Mobile Documents/com~apple~CloudDocs/ondarock/node_modules/strapi/lib/services/entity-service.js:83:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:35420) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:35420) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

And this is how I set up the artitsts Collection type:

./api/artists/models/artists.settings.json:

{
  "kind": "collectionType",
  "collectionName": "artists",
  "info": {
    "name": "artists"
  },
  "options": {
    "increments": true,
    "timestamps": true,
    "draftAndPublish": true
  },
  "attributes": {
    "name": {
      "type": "string",
      "required": true
    },
    "records": {
      "via": "artists",
      "collection": "records"
    }
  }
}

Any idea on how to overcome this ValidationError?

2- How to import pre-existing relations. The voiceover from the linked YouTube video seems to suggest the same approach implemented in this suggestion: Need Help: How to add (if not exists) related records in POST Method - #4 by DMehaffy by @DMehaffy, which is very helpful, so I think I can basically copy and paste it even though I am using bootstrap.js and not a POST method. I am blocked by 1 on actually trying it though.

One question: I’m much more familiar with SQL than I am with JS/React. If I just manually write a SQL procedure to populate the relations table (artists_records__records_artists because it’s a many-to-many relation) in the Strapi db, by using the existing ids from the php application and joining them with the Strapi ids across databases, would that work as well? Again, this is a one-off process so I just need to get the content on the other side.

Thanks in advance to anyone who can offer thoughts.

Roberto

Inside foreach, replace data.name with entity.name. Also, wrap the code with trycatch.

Thank you! What a silly mistake.

Also, in case it’s interesting for anyone else: selecting into the tables that encode relations directly in mysql seems to work well, this allows to encode relations after importing the non-relation data using lazurey’s plugin.