The piece of code belows downloads a file and compares it with itself using the SHA algorithm in order to prove the download was correct. I tested it with a number of binary files that were available on my Windows box and always worked right.

use strict;
use LWP::UserAgent;
use Digest::SHA1;

my $srcfile = "c:/...../cab/WBCust.CAB";
my $tempdl  = "temp.tmp";

my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new(GET => 'http://127.0.0.1/...../cab/WBCust
+.CAB');
my $r = $ua->request($req)->content;

open F, ">$tempdl" or die "$! $tempdl";
binmode F;
print F $r;
close F;

computeSHA( $srcfile );
computeSHA( $tempdl );
unlink $tempdl;
exit;

sub computeSHA() {
    my ($srcfile) = @_;
    my $sh_pre = Digest::SHA1->new;
    
    open S1, $srcfile or die "$! $srcfile";
    $sh_pre->addfile(*S1);
    close S1;
    
    print "$srcfile:\n " . $sh_pre->hexdigest . "\n";
}
[download]

c:/...../cab/WBCust.CAB:
 3816f34e861b6268c716b9f06091dee0d580f2cb
temp.tmp:
 3816f34e861b6268c716b9f06091dee0d580f2cb
[download]

Thanks everyone for you help, you've all given me very useful advice!

Thanks
Chris

OK it looks like I'll be downloading a 45MB file, is LWP ok/the right choice to be downloading this file?

Thanks

Chris

What if it drops the connection halfway through on a slower link? You will have to cater for that, like resuming a download at a given position, but not all servers allow that.

I'd say it's best to yourself a download manager, like getright, netant, etc, that are specialized in downloading large files efficiently from the Internet. They can do special speed optimization too, like openning multiple connections and download sections of the file simutaneously.

According to the LWP::UserAgent docs (verbatim copy-and-paste):

$response = $ua->get('http://search.cpan.org/',
                               ':content_file' => '/tmp/sco.html'
                            );
[download]

use WWW::Mechanize;
[download]

Abigail

How would that help save the file?

Abigail-II gave a good advice to use the WWW::Mechanize module from CPAN. WWW::Mechanize does allow downloading/saving of a file easily (as with LWP::UserAgent).

The WWW::Mechanize module was derived from the LWP::UserAgent module, which has a mirror function to save/download a file:


$ua->mirror( $url, $filename )
[download]

If you go to the examples of the WWW::Mechanize module, you will find an example program from merlyn that 'walks' a site and 'suck down' all the images, which would apply to files in this case.

I have got the logging in part done with the code shown below, but how do I download the file and save it? The example I show is more for downloading a web page. Anyone have any ideas?

$ua->credentials($host, $realm, $user, $pass);
[download]

$ua->mirror($url, $file);
[download]