As tye pointed out, using threads or forks is a way to keep things moving. While some processes hang, others can continue. As the slow processes time out, they are cleared away to make room for new processes.

And you definitely want to put a throttle on this, especially if you are talking about potentially hundreds of processes!

You might want to consider merlynīs parallelizing engine which is mentioned in this informative perlmonks discussion, as well as his forking parallel link checker column which he mentioned in response to some similar code submitted by kschwab earlier.

Matt

P.S. I was going to dust off my so-called "Hatchery" module from my non-released bag of tricks, but considering Randallīs columns I think it would be pretty redundant. sigh.It is amusing to note that what he called "hungry puppies" I tended to call "screaming kids", although I did refer to them as a litter. Well, perhaps once I finish my trip Iīll polish it up and post a comparitive analysis if there seems to be enough differences to warrant it.