file copy and net usage

smackdab has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

Looking for some advice: I want to copy multiple text files from a few servers to my local PC.

This is on Win32, so I guess I'll be doing some: Win32::API::CreateFile()...ReadFile()..WriteFile() type stuff.

I want to be able to limit the network usage (depending where I am and how "impatient" I feel). I figured I'd spawn off a few processes and then have each one start copying a file. Now to limit the bandwidth, I was thinking of setting a Time::HiRes sleep timer and then awake and copy some amount of bytes...

Any advice on things to consider before I "jump" in??? (how long to sleep or how much to copy?)

Any advice on how to specify the rate? (Does "each child process can use approx 10% of the bandwith work?") (At work it could be ethernet and home could be a modem or cable modem...)

I am going to have a config file that states the network speed as I don't want to figure out how to automatically determine it ;-)

Thanks for ANY thoughts !!!

Comment on file copy and net usage

Replies are listed 'Best First'.
Re: file copy and net usage by Cine (Friar) on Aug 03, 2002 at 09:44 UTC
Use Inline::C to implement the SetQoS call from http://msdn.microsoft.com/code/default.asp?url=/code/sample.asp?url=/MSDN-FILES/026/002/258/msdncompositedoc.xml Update: If forgot to include the link to the API... http://msdn.microsoft.com/library/default.asp?url=/library/en-us/qos/gqosstruct_4nfn.asp T I M T O W T D I	[reply] [d/l]
Re: Re: file copy and net usage by smackdab (Pilgrim) on Aug 04, 2002 at 01:57 UTC
Thanks, I didn't know this. It looks like it only supports WinSock, so I think I will need an agent on the server to communicate with. If I do this, I can also add compression, which would be nice. Not sure if I want an agent right now....	[reply]
Re: file copy and net usage by Courage (Parson) on Aug 03, 2002 at 09:56 UTC
Interesting question, and I really want to hear a comprehensive answer. As of my personal opinion, here is my vision on where to start: wget utility from cygwin package has option `--limit-rate=RATE` which limits download bandwidth. Perform some RTFS if you're familiar with C as to current load of modem/eternet-card is a question where it is hard to give a correct answer, because it is hard to distinguish network load from one to another. I mean, say, your 100MBit ethernet card loaded by 5% because of small traffic, but then you connect to 256Kbit Internet, which is overloaded by outside download. What I would recommand anyway - get information from `netstat` command with different parameters, and your script to decide what to do with certain statistical results... may be just use `wget --limit-rate=[rate-specify-here]` command from your script? I would be hapy to see perl answer to the first item. Courage, the Cowardly Dog.	[reply] [d/l] [select]