The latest I've tried this on is 5.18, so it might have been fixed in later builds

It hasn't been - and it goes back to at least 5.16, and it affects 32-bit perls in the same way for unsigned values greater than 2 ** 31.

I couldn't find a way to successfully perform integer division/modulation with these large unsigned values without resorting to XS (as you've done).
(Update: Obviously, I've ignored Math::BigInt.)
However, I'm sure the integer pragma is supposed to cater for this and I couldn't find anything in the docs to suggest otherwise.

So I've submitted a bug report against integer for this - in case there's someone who cares enough to fix it.
(I hit "Send" before thinking to reference the original PM post in that report ... sorry 'bout that.)

Cheers,
Rob