in reply to LWP::UserAgent and HUGE FILES

If head() doesn't work... can't you use the callback interface? That way, you'll only read in blocks of a few k at a time, which you can then discard.

The synopsis for LWP::UserAgent says the basic syntax is

$response = $ua->request($request, \&callback, 4096);
with &callback a sub receiving the data, and 4096 the chunk size.