Looks look like a huge battle! Hmm, if you put (1..1000) in at the start, then three threads start eating them, then you start adding more (1..1000) here's what I'm thinking: Perl never deallocates memory, so it's not recycling memory in the way you want, but you /are/ adding a list of 1000 integers onto another list (very fast too, I might add), so I'm really wondering what are you expecting? A fixed-field array of 16-bit integers stored linearly in memory will take ~16KB on it's own, and in the case of perl, that will include the overhead of a perl primitive for each element, plus the indexing of the array. I think you've chosen a method to beat the hell out of yourself; is there any other way you might get this done?

Are your processing rates really at this ratio? Or is it simply an example?

Update: Thread::Queue has been mentioned before, but how about this: you make a queue for each sub, and pass work to it that way. Just have the 'master'/'manager' round-robin the work, then maybe your queues (read: memory usage) will be smaller.

mhoward - at - hattmoward.org