in reply to
LWP Parallel CPU Usage
Have you tried fetching the content of just 50 sites at a time? Perhaps 2000+ is too aggressive for your hardware.
Comment on
Re: LWP Parallel CPU Usage
Replies are listed 'Best First'.
Re^2: LWP Parallel CPU Usage
by
Anonymous Monk
on Sep 28, 2007 at 00:49 UTC
How do I set how many sites I want to fetch at a time with LWP Parallel? I have both max_hosts and max_req set to 10.
[reply]
In Section
Seekers of Perl Wisdom