Introduction

Discipulus will never be a master: he likes too many things to become a master in one of them. He likes Perl. Here at the monastery he looks at tchnologies shown by monks, often like Linus watching russian names reading Dostoyevsky..

He looks at Perl parallel programming with a big interest: he suspects parallel programming will be one of major battlefield for next generations of programmers. He'd like these programmer to be Perl ones.

He has his cell carpeted of good examples of parallel programming. He also knows it is important to choose right masters to learn and follows with a big attention all parallel programming examples by, among others, BrowserUk and zentara.

Then it happened that marioroy joined the monastery: his work, MCE - Many-Core Engine for Perl hit Discipulus's curiosity strongly: perhaps with such tool even he can, simply and safely produce some decent parrallel Perl program? Infact he's a bit lazy and many working examples are difficult to adapt without falling into horrible pitfalls.

So Discipulus, taken his curage in his hands, has gone knocking at BrowserUk's and zentara's doors to ask their opinion...

(please note that I confuse parallel programming terminology, still. So when I say multithread you can read parallel: the dialogue is interesting beside my grossolane errors.

The Dialogue

Discipulus:I follow your posts about parallel multithreading programming since ever and I follow them as a pupil (discipulus in latin..) follow his masters. I say this withut any flattery intention. As you stated somewhere (iirc) multithreading in Perl is difficult to realize but is possible to do as you shown many many times.
What i will be glad to hear from you is an opinion about Perl MCE by marioroy available at http://search.cpan.org/~marioroy/MCE-1.827/ and visible in many posts by marioroy here in the monastery.

BrowserUk: I mostly don't recommend MCE, (though I have suggested people consider it a few times), simply because I don't us it personally.

zentara: I havn't the time to test MCE out. Besides I would rather do it manually, since I have total control of the code, as in Reusable threads demo.

Discipulus:Now I know you do not need such a thing like MCE to produce good multithread perl programs. but what about us? If i have the need or the curiousity to start write a threaded application i will follow your footsteps: i go to library, i search one of miles of your working complete examples, the closest to my need and i will expand it as my needs. But MCE seems a safe a clean alternative to have a multithread program without writing it from scratch.

zentara:Another thing, Don't confuse forks and threads. MCE is based on forks, from what I can gather, not threads. It gets more confusing because on MS Windows, all processes are threads, not forks.

BrowserUk:I don't use it personally be a) for the most part I don't need it; b) I have a very strong aversion to frameworks. But please note, I saying that I do not generally actively recommend MCE to people, I am not saying that I recommend against it.

Discipulus:MCE is an alternative if you accept the overhead of using a framework: but this is very common in programming: was possible to write webpages without CGI some years ago, writing a server too but CGI was a cheap overhead in front of many pitfalls possible doing all by hand. The same for DBI..
Is MCE something valuable in such perpsective? Is MCE a safer way to do parallel programming in perl, if you accept the minimal overhed and his learning curve?
Fast is fast, i rerun many bechmarks proposed at PM by marioroy and i can confirm. But is solid rock? this is behaind my capabilities to tell. It introduce other pitfalls?
I suspect MCE was highly undervaluated by the perl community and i think multithread is a key field for programmers of next generations: i'd like to see perl kicking competitors in the future.

BrowserUk:If you have an application for multi-tasking that MCE has a widget for, and especially if you can attract the attentions of MarioRoy to construct and refine some examples for your purpose (and perhaps tweak it internally to address any shortcomings your application throws up) then you could do a lot worse. From a cursory inspection, the code seems well designed, constructed and documented, and the author seems to have the time and energy to support you.
Beyond that I really don't have a lot to add.

zentara:As far as the threads controversy goes, these are my thoughts.

0. Threads tend to blow up unexpectedly, so don't use them for anything where you demand certainty and stability. Also many modules are not thread-safe,i.e. Tk

1. Only use threads if you need to share data in realtime between threads in an easy manner, with threads::shared. Otherwise forking a new process is more efficient because you do not have to copy the parent thread's code into the spawned thread, and the process totally frees it's memory when finished.

2. If you are going to use workers, the MCE module looks alright, although it reminds me of Parallel Fork Manager. If I was to chose the one foolproof way of doing the task of many workers sharing data in realtime, I would use Parallel::Fork::Manager along with SYSV Shared Memory segments. (zentara unfold a perchment with a wonderful example.. )

Discipulus: the answer you gave me was somehow expected and somehow unwanted: I know you have no need for such framework because you can write down the right way a multithread perl program using perl and it's basic tools.
But i've learn to not ask questions if i'm not prepared to receive back every answer :=)
I also understand your avversion to frameworks in general: why be constrained into a frame when my puissance can spread out free and untouched? this is a declination of the hubrys in it's positive, perlish sense.
But hubrys without solid rock foundations is foolishness and is where frameworks are valid options: I pay some of my freedom to the frame but i'm almost sure i'm not shooting on my own feets.
In this perspective i consider MCE a true gem: can MCE open the way to multithread Perl programming to many,many more peoples? I think this is it's virtue and this merit i suspect was underestimated.
I suspect and hope MCE can help perl programmers to afford such complex thing as multithread programming is.
If so we must spend some energy to push MCE forward and suggesting it as a way to start, safely, to do multithreaded perl programs.

(saying the above I not mean MCE is the only way to do it right with Perl parallel programming. As always TIMTOWTDT and any author can choose among many.)

Note that I've ask both monks the permission to reproduce parts of our mails: if i've put something in wrong order, if something sounds not to be the real opinion of such monks it is my complete fault and i'll correct

L*