Stream buffer data stored in database memory leak

System Information
  • Strapi Version: 4.1.12
  • Operating System: OSX
  • Database: MySQL
  • Node Version: 14
  • NPM Version: 6.14

Hi,

When uploading assets the stream buffer is being stored in the database for each format. This causes a huge record size.
There has been a possible solution for this: Can we remove media formats stream from core API Responses? - #3 by arthur-eudeline
But it doesn’t work entirely. Because when replacing media, the error still exist.

How do you guys tackle this problem? Does everybody just accept the huge database storage?

Example part in database:
Table: Files
Column: Formats

{
  "small": {
    "ext": ".jpg",
    "hash": "small_pexels_pixabay_2159_5f254c27eb",
    "mime": "image/jpeg",
    "name": "small_pexels-pixabay-2159.jpg",
    "path": null,
    "size": 15.67,
    "width": 500,
    "height": 321,
    "stream": {
      "fd": 49,
      "end": null,
      "mode": 438,
      "path": "/var/folders/gc/jh82fwnd5b1_dkw4sxvr_gyr0000gn/T/strapi-upload-wom7nM/small_pexels_pixabay_2159_5f254c27eb",
      "flags": "r",
      "closed": false,
      "_events": {},
      "autoClose": true,
      "bytesRead": 15669,
      "_eventsCount": 1,
      "_readableState": {
        "sync": false,
        "ended": true,
        "pipes": [],
        "buffer": {
          "head": {
            "data": {
              "data": [
                255, 216, 255, 219, 0, 67, 0, 6, 4, 5, 6, 5, 4, 6, 6, 5, 6, ..... etc ... etc more than 130k of numbers
              ],
              "type": "Buffer"
            },
            "next": null
          },
          "length": 1
        },
        "closed": false,
        "length": 15669,
        "decoder": null,
        "errored": null,
        "flowing": null,
        "reading": false,
        "encoding": null,
        "destroyed": false,
        "emitClose": true,
        "endEmitted": false,
        "objectMode": false,
        "autoDestroy": false,
        "dataEmitted": false,
        "readingMore": false,
        "closeEmitted": false,
        "errorEmitted": false,
        "needReadable": false,
        "highWaterMark": 65536,
        "defaultEncoding": "utf8",
        "emittedReadable": false,
        "multiAwaitDrain": false,
        "resumeScheduled": false,
        "awaitDrainWriters": null,
        "readableListening": false
      }
    }
  },
  "medium": {
    "ext": ".jpg",
    "hash": "medium_pexels_pixabay_2159_5f254c27eb",
    "mime": "image/jpeg",
    "name": "medium_pexels-pixabay-2159.jpg",
    "path": null,
    "size": 33.52,
    "width": 750,
    "height": 482
  }
}

Hello :slight_smile: Is it by uploading through the API or using the Media Library in the Admin Panel?

What provider are you using? Is it provider-upload-do? If yes, then you may want to check this issue : Image upload persist buffer object in the database causing performance issue · Issue #15154 · strapi/strapi · GitHub
It seems related to the provider itself that Strapi doesn’t manage.