This topic was discussed in this StackOverflow post some time ago, which also refers to this perldoc entry (perlfaq3).

It can be problematic.   You definitely want to set up your own application-specific repository, as (say) any Google search on “Perl in shared hosting” will discuss at great length, so that any packages which you install will be installed there and, once installed there, will be seen in preference to the system environment’s set of global packages (which you should not touch).   Things can get interesting since the explicit request to install one package might install its pre- and co-requisites, and sometimes CPAN will decide that system libraries satisfy some of those dependencies ... while a different system might have a different set.

When you install on the new system, use the list that you have built to re-install the packages on the target system ... allowing CPAN to deal with all of the requisites and, of course, to install the binary libraries that are correct for the new environment.   As others have already said, you cannot reliably just copy the directory’s contents verbatim, because there’s a lot more stuff in there than just Perl source-code.

More rare – but I do still encounter it from time to time – is software that is actually dependent upon a certain version of a module such that it won’t work properly with a later one.   Anytime you discover this, you should raise the issue within the project team so that (if practicable) the software can be re-written to remove it.