in reply to Re: WWW:Mechanize click_button stream to file?
in thread WWW:Mechanize click_button stream to file?
LARGE DOCUMENTS
If the document you want to fetch is too large to be kept in memory,
then you have two alternatives. You can instruct the library to write
the document content to a file (second $ua->request() argument is a file
name):
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
my $req = HTTP::Request->new(GET =>
'http://www.linpro.no/lwp/libwww-perl-5.46.tar.gz');
$res = $ua->request($req, "libwww-perl.tar.gz");
if ($res->is_success) {
print "ok\n";
}
else {
print $res->status_line, "\n";
}
Or you can process the document as it arrives (second $ua->request()
argument is a code reference):
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$URL = 'ftp://ftp.unit.no/pub/rfc/rfc-index.txt';
my $expected_length;
my $bytes_received = 0;
my $res =
$ua->request(HTTP::Request->new(GET => $URL),
sub {
my($chunk, $res) = @_;
$bytes_received += length($chunk);
unless (defined $expected_length) {
$expected_length = $res->content_length || 0;
}
if ($expected_length) {
printf STDERR "%d%% - ",
100 * $bytes_received / $expected_length;
}
print STDERR "$bytes_received bytes received\n";
# XXX Should really do something with the chunk itself
# print $chunk;
});
print $res->status_line, "\n";
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^3: WWW:Mechanize click_button stream to file?
by Qiang (Friar) on Feb 13, 2007 at 19:17 UTC | |
by Anonymous Monk on Feb 14, 2007 at 09:34 UTC | |
by Qiang (Friar) on Feb 14, 2007 at 15:46 UTC |