in reply to
Get 10,000 web pages fast
I would split the list into 100 files, and launch 100 instances of wget or curl, and let them do the mirroring.
Comment on
Re: Get 10,000 web pages fast
In Section
Seekers of Perl Wisdom