I got sidetracked in the parent post. The OP asks about C pointers stored in an SV as numbers, not as references.

The answer is quite simple: It doesn't matter whether it's stored as an IV or as an UV since the value is casted to the correct pointer type before anything ever sees it as a number.

By the way, that code will work whether it's stored as an IV or as an UV.