Upload Buffer using strapi upload

Community Packages Plugins
1
System Information
  • Strapi Version: 4.1.9
  • Operating System: MacOS
  • Database: Postgres
  • Node Version: 16
  • NPM Version: 6.14
  • Yarn Version: 1.19.1

Hi,

I create a pdf file server side in a controller and want to upload it using my strapi upload provider.

In Strapi V3, I used to do it calling the upload function from upload plugin directly, using an object containing all data, like that :

const uploadAndLinkDocument = async (document, name, refId, ref, field) => {
  // add generated document
  const uploadService = strapi.plugins.upload.services.upload;

  // Transform stream files to buffer
  const parts = await toArray(document);
  const buffers = parts.map((part) =>
    _.isBuffer(part) ? part : Buffer.from(part)
  );

  const buffer = Buffer.concat(buffers);

  const data = {
    fileInfo: { name },
    refId,
    ref,
    field,
  };

  await uploadService.upload({
    data,
    files: {
      name,
      buffer: true,
      path: buffer,
      type: "application/pdf",
      size: document.size,
    },
  });
};

But in strapi v4, this fails because the enhanceFile method from upload controller tries to read file.path as it’s a real file. Line 118 :

fs.createReadStream(file.path);

So this throws an error with a buffer.

Does anyone knows how to use strapi upload provider to upload a generated Buffer ?

Thanks !

3 Likes
2

If anyone needs that, here is the solution that I implemented (based on strapi upload plugin code)

It works well in Strapi v4

const { Readable } = require("stream");

const getServiceUpload = (name) => {
  return strapi.plugin("upload").service(name);
};

const uploadAndLinkDocument = async (buffer, {filename, extension, mimeType, refId, ref, field, user}) => {
  const config = strapi.config.get("plugin.upload");

  // add generated document
  const entity = {
    name,
    hash: filename,
    ext: extension,
    mime: mimeType,
    size: buffer.length,
    provider: config.provider,
  };
  if (refId) {
    entity.related = [
      {
        id: refId,
        __type: ref,
        __pivot: { field },
      },
    ];
  }
  entity.getStream = () => Readable.from(buffer);
  await getServiceUpload("provider").upload(entity);

  const fileValues = { ...entity };
  if (user) {
    // created_by has a foreign key on admin_users. Here our user is a regular user, so it fails.
    // uncomment this only if the user you pass to the function is a strapi admin.
    /*fileValues[UPDATED_BY_ATTRIBUTE] = user.id;
    fileValues[CREATED_BY_ATTRIBUTE] = user.id;*/
  }

  const res = await strapi
    .query("plugin::upload.file")
    .create({ data: fileValues });
  return res;
};

ref is strapi reference to a model, like “api::model.model”

Hope this helps

2 Likes
3

Hi!

First of all, thanks, the upload part works great!
But setting the relation just wont work for me…

SqliteError: insert into `files_related_morphs` (`0`, `1`, `10`, `11`, `12`, `13`, `14`, `2`, `3`, 
`4`, `5`, `6`, `7`, `8`, `9`, `file_id`, `related_id`, `related_type`) values ('p', 'r', 'c', 't', 
'u', 'r', 'e', 'o', 'f', 'i', 'l', 'e', '_', 'p', 'i', 19, '98', 'plugin::users-permissions.user') 
- table files_related_morphs has no column named 0
    at Database.prepare (...node_modules\better-sqlite3\lib\methods\wrappers.js:5:21)

This is my code:

const uploadAndLinkDocument = async () => {
                        const config = strapi.config.get("plugin.upload");
                    
                        const entity = {
                            name: `useravatar-${event.params.data.username}`,
                            hash: event.result.id,
                            ext: ".jpg",
                            mime: "image/jpeg",
                            size: response.data.length,
                            provider: config.provider,
                        };
                        entity.related = [
                            {
                                id: `${event.result.id}`,
                                __type: "plugin::users-permissions.user",
                                __pivot: "profile_picture",
                            },
                        ];
                        entity.getStream = () => Readable.from(response.data); 
                        await strapi.plugin("upload").service("provider").upload(entity);
                    
                        const fileValues = { ...entity };
                    
                        const res = await strapi.query("plugin::upload.file").create({ data: fileValues });
                        return res;
                    };

Do you perhaps have a clue what’s wrong here?

If I remove __pivot: "profile_picture", the relation almost gets set correctly:
Screenshot_1
Only field and order columns are null.

1 Like
4

Figured out what the error was. I had to change the following:

entity.related = [
  {
     id: `${event.result.id}`,
     __type: "plugin::users-permissions.user",
     __pivot: { field: "profile_picture" }
  },
];
5

I registered to tell you a huge Thank you!
I had a task to upload images from the external resource and create entity with it. And I had urls for images. My idea was to pass stream to the upload module without saving it to the hdd. And your example of using provider helped a lot. I debugged upload module and also found the provider but had no luck to use it in a right way… Thank you!
Kind of my result:

const uploadProvider = strapi.plugin('upload').service('provider');
const config = strapi.config.get('plugin.upload');
const { photos, ...jsonData } = record;
const media = await Promise.all(photos.map(async photo => {
  const file = await axios.get(photo.url, {
    responseType: 'stream',
  });

  const fileNameNoExt = path.basename(photo.name, path.extname(photo.name));
  const entity = {
    name: `${jsonData.name} ${fileNameNoExt}`,
    hash: `${slugify(jsonData.name)}_${fileNameNoExt}`,
    ext: path.extname(photo.name),
    mime: mimeTypes.lookup(photo.name),
    size: file.headers['content-length'],
    provider: config.provider,
    getStream: () => file.data,
  };

  await uploadProvider.upload(entity);
  return strapi
    .query('plugin::upload.file')
    .create({ data: entity });
}));

try {
  return await strapi.entityService.create('api::product.product', {
    data: {
      ...jsonData,
      media,
    },
  });
} catch (e) {
  ...
}
6

version 2, with image optimization (unfortunately the stream is replaced by buffer)

const bytesToKbytes = (bytes) => Math.round((bytes / 1000) * 100) / 100;

const createAndAssignTmpWorkingDirectoryToFiles = () => fse.mkdtemp(path.join(os.tmpdir(), 'strapi-upload-'));

const media = await Promise.all(photos.map(async photo => {
  const file = await axios.get(photo.url, {
    responseType: 'arraybuffer',
  });

  const fileNameNoExt = path.basename(photo.name, path.extname(photo.name));
  const entity = {
    name: `${jsonData.name} ${fileNameNoExt}`,
    hash: `${slugify(jsonData.name)}_${fileNameNoExt}`,
    ext: path.extname(photo.name),
    mime: lookup(photo.name),
    size: bytesToKbytes(Number(file.headers['content-length'])),
    provider: config.provider,
    tmpWorkingDirectory: await createAndAssignTmpWorkingDirectoryToFiles(),
    getStream: () => Readable.from(file.data),
  };

  await strapi.plugin('upload').service('upload').uploadImage(entity)

  return strapi
    .query('plugin::upload.file')
    .create({ data: entity });
}));
3 Likes
7

Thank you so much, I will probably make a PR to put this into the upload plugin natively.

2 Likes
8

First of all, thank you @Yuriy_Zaitsev for this. It’s working like a charm.
@Cpaczek Did you already open a pull request for this?

9

thanks for that, wouldn’t have managed otherwise. I came across this too which was helpful:

10

Thanks for your answer.