in reply to Speeding up/parallelizing hundreds of HEAD requests

This situation could be viewed as a cacheing issue, and addressed that way, without delving into the complexities of parallel processing.

If the source data is relatively static, and the requests are "browsing" style, then you potentially are retrieving the same data periodically, and cacheing would help.

The simplest would be to use the Memoize module. Minimally more work, but more accurate would be to use Memoize::Expire.

     "As you get older three things happen. The first is your memory goes, and I can't remember the other two... " - Sir Norman Wisdom

  • Comment on Re: Speeding up/parallelizing hundreds of HEAD requests

In Section Seekers of Perl Wisdom