I suspect that the setting of the maximum burst rate to 10k (10240) here (in new()):

    my $mbps = $arg{MegabitsPerSecond} || croak "No MegabitsPerSecond 
+passed";
    my $bps = $mbps * 1024 * 1024 / 8;

    my $bucket = new Algorithm::TokenBucket $bps, 10240;
[download]

Combined with the minimum accounting period of 1/100th of a second here (in get()):

        sleep 0.01 until $bucket->conform($len);
        $bucket->count($len);
[download]

Means that the maximum throughput possible is 10kbytes per 0.001 of a second = 1000KBytes or (roughly) 10Mbps.

Throw in the out-of-band overheads (writing to the local file), and that probably explains why it has a fixed upper limit regardless of your requested limit.

Two (3) possible solutions:

1. Increase the max-burst value.

   Try doubling it and see what through put it gives you. Adjust accordingly.

2. Decrease the accounting period.

   Try halving it. Adjust ...

3. A mix of the two.

Also, 4096 byte blocksize seems very small for transferring large files across a network? I try 64k and see what affect that has.

Thanks BrowserUK!

With the max-burst doubled, I was seeing better throughput. So I doubled it again and it looks like now I'm seeing around 50mbps

Regarding the block size, I thought the same, however it seemed that anything over 4K was not helping, and I only got better throughput by decreasing it to 1024 and finally 512. Perhaps now that I've increased max-burst, a larger block size wouldn't hurt?

at least this is working now; I wonder if Net::FTP::Throttle should be updated to increase that max-burst? How does stuff like that get done, should I be prompting someone or going at it myself?

kind regards, Joe

How does stuff like that get done, should I be prompting someone or going at it myself?

I'd start by sending the author an email pointing to this thread at perlmonks and go from there.

With regard to blocksize. I just did some not-very-representative testing using the loopback address on my machine and with 4k blocks the maximum throughput I can get is 140MB/s:

C:\test>sock-rated.pl -O=4 -I=4 1000
svr: request for 1048576000 
1048576000 took 6.999 seconds. [142.89 MB/s]
[download]

By using 100k blocks, I can get that up to over 300MB/s:

C:\test>sock-rated.pl -O=100 -I=100 10000
svr: request for 10485760000 
10485760000 took 33.032 seconds. [302.74 MB/s]
[download]

Not a real-world test, but still 4k seems like a pifflingly small amount.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?

joetesta,

On a lan without going through a router, large block sizes will help, but once you use the Internet, the large blocks will cause fragmentation. Try using 1492 which is the most common packet size for routers on the Internet. If you are on a GigE Internet connection, I believe 14,920 can be used, but realize the first router that can't support that size will cause fragmentation.

Also realize that once more than one down-load is in progress, you may go over your bandwidth limit. On all of my Internet facing servers, I use the open source 'pure-ftpd' which allows bandwidth to be throttled by user, group, time of day, etc. You can chain-root the ftp session.

If you can't chance the server, then try to limit the number of concurrent ftp sessions and limit each individual ftp session to a value less than the max usage divided by the number of concurrent sessions.

Good Luck!

Hello All, I am using the Net::Ftp for file transfer and was wondering if there was any way to display the throughput of the transfer using some FTP commands? Thanks in advance ! Arun