ezekiel has asked for the wisdom of the Perl Monks concerning the following question:

I am using LWP::UserAgent to retreive files from a FTP site. Everything appears to work fine until I attempt to get a file that is supposedly 65M in size. At this point the script dies with error:

"Out of memory during "large" request for 134221824 bytes, total sbrk() is 271669258 bytes at .."

The code I am using is below. I know the physical memory is available. Any thoughts on how to get around this problem? Thanks in advance.

my $ua = LWP::UserAgent->new(); my $response = $ua->get("$url"); if ($response->is_success) { my $content = $response->content; open (OUTPUT, ">out.txt") or die "Cannot open the output file: $!" +; print OUTPUT "$content"; close (OUTPUT) or die "Cannot close the output file: $!"; } else { die $response->status_line; }

Replies are listed 'Best First'.
Re: Out of memory while using LWP::UserAgent
by Fletch (Bishop) on Jan 04, 2006 at 21:28 UTC

    See the section "Large Documents" in perldoc lwpcook.

Re: Out of memory while using LWP::UserAgent
by ambrus (Abbot) on Jan 05, 2006 at 12:22 UTC

    Search for content_cb and content_file in perldoc LWP::UserAgent. This shows how you can write the data to a file or process it with a callback instead of writing to a perl scalar.

Re: Out of memory while using LWP::UserAgent
by kwaping (Priest) on Jan 04, 2006 at 22:24 UTC
    Maybe Net::FTP would better suit your needs?