in reply to Re: parallel programming and MCE - hubris and the frame
in thread parallel programming and MCE - hubris and the frame

Finally, most of us, have single core or a few core computers. On machines like these, parallel processing is usually just an exercise in coding, and dosn't really speed anything up.

Surely this depends on the task?

Parallel::ForkManager has long given the simple example of a script to download content from many websites. Here obviously the program is not bound by disk I/O but by network response, and parallelization results in many-fold speed increases. I am using MCE for similar tasks and "dosn't really speed anything up" could not be further from my experience.

I have switched recently from using Parallel::ForkManager to using MCE, mostly because of wanting the extra features MCE provides such as easy shared data structures with MCE::Shared, the fact that the author is very committed to supporting all major Oses, and the fact that the author is very quick to support and help.

Yes, I am a little mindful of the dangers of committing to a "framework", but as brother Discipulus says, we all use frameworks. Some have not been a good choice over my years in this work, but the vast majority of the ones I have chosen to use, after careful research and experimentation, have been reliable and powerful and useful. Perhaps Perl's prioritization of not breaking backwards compatibility is to thank for some of this, as well as the character of Perl developers and the Perl community.

Overall, MCE has been a great choice for me, your basic common-or-garden journeyman Perl developer (I'm not surprised you get different opinions from computer scientists like BrowserUk and zentara).


The way forward always starts with a minimal test.
  • Comment on Re^2: parallel programming and MCE - hubris and the frame

Replies are listed 'Best First'.
Re^3: parallel programming and MCE - hubris and the frame
by zentara (Cardinal) on Apr 10, 2017 at 14:43 UTC
    Hi, buried in my first reply was "Of course, frameworks like MCE, parallel forks, and spawning worker threads are still very useful in certain circumstances,", so I stand corrected and you are correct.

    But I will have to agree with BrowserUk, that complex frameworks take alot of time to master, and often require the expert guidance of the module author, as shown in Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run. Without the author's guidance, I never would have been able to work that problem out. Also, I use Linux solely, and don't really care if stuff runs properly on Windows or not. I will use what is simple and easy for linux. I do applaud MCE's author for his effort though, it appears to be a great module.


    I'm not really a human, but I play one on earth. ..... an animated JAPH