For a very quick fix to avoid upsetting your generous benefactor, you could simply add a select delay into the while loop in /lib/Net/FTp.pm (~ line 488). I'd put it just before the line (~490)

   select  undef, undef, undef, 0.005; ## Adjust. > slower; < faster.
   last unless $len = $data->read($buf,$blksize);
[download]

You can then adjust the value manually to strike a balance between your needs and theirs.

Adding a more formal parameter that allowed the rate to be specified in terms of kb/second wouln't be too hard to do for your system using a little math and some fudge factor.

Making that generic enough to offer it as a patch would be considerably harder as you would have to incorporate some means of self calibration across systems with different performance. And once it became a part of the standard interface, woe betide the author if it only achieved 1.9kb/sec when requested for 2kb/sec.

Well, if you aren't already, compress the file prior to downloading it. Then use as you suggested, wget to limit the bandwidth. Read the source of wget to see how it is done then implement it in perl if you really need it in perl.

Setting the blocksize on Net::FTP won't make much difference at all.

Not much else to tell ya

Jason L. Froebe

Net::FTP supports a BlockSize setting that might have an effect if set very low. But I think that's doubtful.

I think my first try would be to call get with a filehandle as my local file (get supports either a file name or a filehandle). You could then presumably tie the filehandle to a homegrown object that implements a WRITE method that limits bandwidth by going to sleep periodically or something. Completely untried, but that would be fun to give a shot ...

Another approach is to think about whether you really have to copy the whole file every day. Using rsync might help your bandwith usage also.

Also, rsync has a --bwlimit=KBPS option.

With answers to these questions we might be able to help reduce the times you need to download it at all, and that might make your benefactors very happy.

A few scenarios come to mind:

1. It's a file which only changes once every 2 days or less often
2. It's a log file or similar and you only care about the last X many lines, where X is much smaller than the total number of lines
3. It's a file which changes all the time and you care about all of it

Let's consider these options:

Run a hashing algorithm over the file (eg md5sum) and download the result. If the hash is the same as the previous day then don't download it.

A disadvantage with this is you may need to ask their sysadmin to assist you somewhat, by giving you sufficent access to run this command, or by agreeing to run a scheduled task every night which creates the hash for you.

Rotate the file more often (daily), so that you only need to download the bits you care about.

This is the hard one. All you can do here is:

• Make sure you compress the file
• Shape your traffic
• Hope they'll still like you.
• Consider whether you can download it every two days late at night, instead of daily, so as to minimise the disruption caused.

Of course, if it is one of the first two, then so long as you compress you file as well; one of those methods should help a lot.

Hope this helps.

jarich

Hi jarich, It's a ZIP file full of images that changes every day. So unfortunately none of these ideas will help with my particular situation, but they're definitely good things to keep in mind!

Neil

http://www.onlamp.com/pub/a/bsd/2001/07/26/Big_Scary_Daemons.html

I don't know of any good way to limit bandwidth consumption from your script.