I have the exact same problem.
I really want to avoid inline casting all over the place; super silly.