in reply to Re: Speeding things up -- LWP::Parallel
in thread Speeding things up
Some fiddling has been done. LWP & HTTP replaced with MOJO. I've now gone from ~50 sec to exmamine 100 pages to 16 sec aka 2 URL/s to 6 URL/s.
I wonder how fast you have to hit Argos before it starts to think it's a DDoS? Anyone got any ideas how I can find out? And I don't mean bash it hard until it squeaks. That's just rude.
|
|---|