The idea you are going with is correct to sort and it works.
But I think you may just have a typo. Did you try it as plural:
"colors.price:asc".
So in a sample GraphQL query, I tried it and it works:
query{
products(sort: "colors.asc"){
id
name
colors{
name
price
}
}
}