Strapi v4 deployed to AWS EBS trying to load resource from localhost

My root/config/middlewares.js is like the one provided, except strapi::session is missing. Adding that line prevents strapi from building with the following error

Middleware strapi::session not found.