Thanks diotalevi, That module (actually called Array::RefElem), does appear to do exactly what I was suggesting. I'll need to pour over Perlguts Illustrated some more to convince myself completely.

I think wrapping it in a tied array will achieve exactly what I been trying for two years to get! In my defense, the name does not exactly leap off the page at me, and even the description is somewhat terse. You'd need to be fairly ohfay with XS/internals to understand it.