Hey. I have the same issue. I also use @strapi/provider-upload-aws-s3 with a private bucket and activated signedUrlExpires. I can see the thumbnails in my Media Library. When I use an image in an article, a signed URL with an expire date is generated. The image is then available and displayed until the time expires. After that it is no longer available.
There must be a function that only generates the URL when the API is queried. Does nobody else have this problem?