We ended up creating a plugin with a custom field and completely ignore Strapi’s own Media Library.
The custom field can be used to browse Canto and pick an asset, and the asset is then stored as JSON into the column of the current content type’s table.
There are some caveats to think about when doing this:
The external library we use (Canto) allows manipulating assets. There is a ton of metadata and even the file itself can be changed, all while retaining the original asset’s ID. We use webhooks to inform Strapi when an asset has changed, and then in Strapi search for it across all content types and update it.
Similar things have to be done when deleting an image from the external library which is still linked in Strapi.
It was a lot of work.