The caveat that modules will be downloaded each time the app is run can be worked around using LWP::Simple's mirror sub intead of get.

Of course, you'll have to provide the output file name to mirror:

my $response = $ua->mirror( $url, $file );
#EG:
$response = $ua->mirror( 'http://search.cpan.org/src/GAAS/libwww-perl-
+5.800/lib/LWP/UserAgent.pm',
                         '/usr/lib/perl/5.8.4/site_lib/LWP/UserAgent.p
+m' );
[download]

By doing this, the LWP module checks to see if the file already exists; if it does, a new copy is only downloaded if the remote file is newer than the local copy.