in reply to Re^3: Parallel processing on Windows
in thread Parallel processing on Windows
GNU parallel has never failed me (under Unix, of course). It is a Perl script using threads and Thread::Queue.
Reading (diagonally) the long discussion cited by choroba as to why the word "discouraged" was used, I did not find real arguments except perhaps that Threads:Shared at some time could not handle the cloning of deep, complex data structures to be shared (as I understand it, now it works) and also that you may not be able to find help.
For me Corion's Re: Parallel processing on Windows suggestion served me well for all my parallel needs. I have used How to create thread pool of ithreads (the posts by BrowserUK in there) as my starting point.
There is also marioroy's MCE which I have never used. It looks solid. See Reusable threads demo on how it is used as an alternative to the threads + Thread::Queue paradigm.
bw bliako
Edit: Another point in the long discussion mentioned above is performance of a thread-enabled perl and also the overheads of creating a new thread. The latter is mostly irrelevant when you follow the model of a pool of workers (the threads' queue) where a number of threads (workers) are created once and then keep processing your data queue. If you don't keep re-creating threads then this point is irrelevant mostly. Then you have the performance of a perl compiled to enable threads which can be really hindered by the various locks put in place to protect you against race conditions etc. in a potentially threaded environment. That penalty is irrespective of whether you use threads or not, it is whether you want Perl to be able to run threads.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^5: Parallel processing on Windows
by Anonymous Monk on Sep 20, 2022 at 21:18 UTC |