in reply to Re: Speeding things up -- LWP::Parallel
in thread Speeding things up

Some fiddling has been done. LWP & HTTP replaced with MOJO. I've now gone from ~50 sec to exmamine 100 pages to 16 sec aka 2 URL/s to 6 URL/s.

I wonder how fast you have to hit Argos before it starts to think it's a DDoS? Anyone got any ideas how I can find out? And I don't mean bash it hard until it squeaks. That's just rude.

  • Comment on Re^2: Speeding things up -- LWP::Parallel