The only downside is that you'd lose the ability to access the outer hash from XSM

Not really, just keep a pointer to the wrapping hash also inside the C structure, as in the OP sample code.