[v4] graphql one-to-one relations performance issue

It’s unfortunate strapi doesn’t keep one-to-one and many-to-one relations in the same table anymore.
I have 200+ fragments to fetch with their related reference id:

{
    fragments {
        id
        attributes {
          modifiers
          content
          join_to_reference_id {
            data {
              id
            }
          }
          reference_id {
            data {
              id
            }
          }
        }
    }
}

The request is very slow and I just need the id.
What would be the best way to optimize my request ?

Create both reference_id of type integer and reference of type relation and only request the reference_id ?