That's pretty close to scenario #2.

So presumably in your script you say something like

use FindBin;
use lib "$FindBin::Bin/lib/";
[download]

My basic suggestion is that you can change it to

use FindBin;
use Config;
use lib ( $Config{archname} =~ /64/? "$FindBin::Bin/lib64" : "$FindBin
+::Bin/lib32" );
[download]

and just ship two complete collections of modules. Then, if you inspect those trees of files and manage to make it so that they don't have any non-identical collisions, you could merge the trees back into a single lib directory. The merge is just nice to save disk space, and not required for supporting both perl versions. If there are collisions, you can decide how much time to spend fiddling with it vs. the size cost of just shipping dual lib directories. "diff -r" can tell you whether any of the perl modules change their installed .pm files when installed under a different arch.

You could even go for a hybrid approach where you install all the non-XS modules under ./lib/, and then install the XS under ./lib32 and ./lib64, and already get most of the space savings without any effort.