in reply to Downloading URL's in Parallel with Perl

Use wget. It's very fast and has a simple command-line interface you can use.

If you can't use that, stick with multi-process Perl. Forking is faster than LWP::Parallel. Use HTTP::GHTTP, or HTTP::Lite if you can't compile C extensions.

  • Comment on Re: Downloading URL's in Parallel with Perl