Just write it directly to a file using LWP::UserAgent and then process the file line by line. According to the docs this has a small memory footprint.

use LWP::UserAgent;

my $ua       = LWP::UserAgent->new;
my $file     = '/tmp/temp.txt';
my $url      = 'http://foo.com/index.htm';
my $request  = HTTP::Request->new( 'GET', $url );
my $response = $ua->request( $request, $file ); 

open FILE, $file or die $!
while (<FILE>) {
.....
[download]

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print