The biggest pain is needing to work out which modules are pure perl and can just be copied over and which have a binary component that need proper installation.

Those in $arch (e.g. i686-linux-thread-multi) vs those that aren't.

Perl can be setup to peek into the libs of previous version, and that's how it determines what's safe and what's not.