Looking at
LWP::Simple sources, it's already done:
sub getstore ($$)
{
my($url, $file) = @_;
my $request = HTTP::Request->new(GET => $url);
my $response = $ua->request($request, $file);
$response->code;
}
And $file ends up in
LWP::Protocol collect() method:
elsif (!ref($arg) && length($arg)) {
open(my $fh, ">", $arg) or die "Can't write to '$arg': $!"
+;
binmode($fh);
push(@{$response->{handlers}{response_data}}, {
callback => sub {
print $fh $_[3] or die "Can't write to '$arg': $!"
+;
1;
},
});
Files are not loaded into memory. You may want to tweak :read_size_hint argument of get() method of LWP::UserAgent if it still overflows.