in reply to LWP::UserAgent and HUGE FILES
I think that the problem comes down to whether the server you are requesting from honours the Range: header. The maz_size parameter uses that to request a the first max_size bytes of the file. If the server doesn't honour this (I believe optional, but I couldn't find the appropriate RFC) header, then you will likely get the lot. You might try the HEAD command, but that isn't always honoured either.
If this is a show stopper to your application, you could consider hacking LWP::Protocol::http.pm/request(). It ought to be possible to cross-reference any Range: header specified on the request and bottle out of the read loop early, or perhaps better, read and discard content greater than requested, but your a brave man if you decide to mess with this stuff:)
If neither the HEAD requests nor the Range: header are being honoured by the server, it might be better to nag the sysops of that server to enable/upgrade them.
|
|---|