If you use LWP::UserAgent, it looks like you can pass the :content_cb option to get and deal with the response a little at a time. If your callback calls die, the request is aborted, so you can quit reading once you've found what you want.

Update with code and output:

use LWP::UserAgent;

sub little_bit {
    my ( $content, $response, $protocol ) = @_;

    printf "chunk length %d\n", length $content;
    if ( $content =~ /the/i ) {
        print "chunk with the 'the': $content\n";
        die;
    }
}
my $ua = LWP::UserAgent->new();
my $response
    = $ua->get( 'http://perlmonks.org/',
                ':content_cb'     => \&little_bit,
                ':read_size_hint' => 100 );
__END__
chunk length 100
chunk length 100
chunk length 100
chunk with the 'the':  The Monastery Gates
    </title>

    <link rel="stylesheet" href="/css/common.css" type="text/
[download]