in reply to LWP Browser->Get Challenge

I actually had a very similar sort of problem a while back, where the files I was trying to download were quite large, but I never managed to resolve it using LWP.

However, while this may not be what you want to do, I managed to resolve my own problem by using curl instead, and setting a limit-rate on the downloads. i.e. I used my same Perl script for everything, except for the downloads I simply issued a system call from my script to kick off curl with the options I provided.

You sound like somebody who may already know about curl, in which case you'll already know there are a gazillion options for it. My downloads were mostly FTP, but it works equally well for http, or whatever.

The option you'd want to use with curl is "--limit-rate".

For example, one of my commands looks similar to this:

$cmd="curl -u $uname:$pwd -O --limit-rate 50K ftp://$host$dir$file"
... where 50K is the limit-rate which I've set, but you can set it to whatever you like (e.g. 10K, 35K, etc).

I'm sure there are also probably curl options for getting past the certificate too.