in reply to Re: Use WWW::Mechanize to Download Pictures of Sayuri Anzu
in thread Use WWW::Mechanize to Download Pictures of Sayuri Anzu

I recommend falling on the side of over-politeness b/c it's more likely to keep resources open and less likely to inspire lots of new tools to block this kind of thing. I usually sleep 30 seconds or more per domain (requests to different sites don't really need sleep). The big cats like Google go even longer with their bots.

  • Comment on Re^2: Use WWW::Mechanize to Download Pictures of Sayuri Anzu