Re^3: Using Perl to run a Windows command-line utility many times with ordered, parallel execution

Thread::Queue would work just as well, though it would consume a little more memory without some additional code to limit the size of the queue.

But for the OPs 10k items and 20 threads that would still come in at less than 40mb.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re^3: Using Perl to run a Windows command-line utility many times with ordered, parallel execution

Replies are listed 'Best First'.
Re^4: Using Perl to run a Windows command-line utility many times with ordered, parallel execution by Jim (Curate) on Apr 17, 2014 at 00:35 UTC
What about Thread::Queue::MaxSize?	[reply]
Re^5: Using Perl to run a Windows command-line utility many times with ordered, parallel execution by BrowserUk (Patriarch) on Apr 17, 2014 at 02:30 UTC
What about Thread::Queue::MaxSize? Honestly? If you're not the author, read on: What a mess! The design makes no sense at all. *Queues are (should be) reliable, ordered and efficient, sync-free buffers between producer and consumer threads.* What point is there in having a queue that dies when it reaches the limit the programmer set? Now the programmer has to both remember what the limit is, and check it before every enqueue. Like having a dog and barking yourself. Completely pointless. And a queue that throw things away (silently or with a warning) is total corruption of the concept. The whole point of a self limiting queue is to provide a self-checking, buffer between producers and consumers that automatically restrains over active producers. Can you imagine using a supermarket checkout that random threw items of the current customer into a bin, whenever a new customer miscounted the belt limit and added their items to the end? (Inherited from the underlying module, but perpetuated.) A queue that allows inserts isn't a queue. The ability to insert and delete items into/from the middle of a queue totally violates the design contracts of a 'queue'. This is why I rejected patching or modifying Thread::Queue. Because the new owner rendered the module inefficient and corrupt when he added those facilities to the original authors correct and efficient implementation. For no good reason beyond 'making his mark'. Even if the design made sense, I would not use a module that is so badly and naively implemented. Where is the sense in allowing case-insensitive, 'string constant' configuration options? If configuration constants make sense, the module should provide proper constants. If there was any merit in allowing case-insensitive options the module should convert them to all upper or all lower on input so that from then on they can be compared directly against hard-coded strings with eq thus avoiding the not inconsequential costs of invoking the regex engine over and over to perform case-insensitive comparisons. Many other nativities. In summary: a very bad design, horribly naively implemented. It typifies everything that is wrong with CPAN! If you are the author. Sorry. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]