Threading a job between multiple machines

SyN/AcK has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
(jeffa) Re: Threading a job between multiple machines by jeffa (Bishop) on Jul 30, 2003 at 15:38 UTC
That's called message passing. You can have threads within a machine, but to access other machines, you need to pass messages. Take a look at MPI (it's a step up from PVM). There are a couple of MPI CPAN modules: Parallel::MPI, but you have to install an implementation of MPI first, i recommend MPICH (MPI Chameleon). Good luck (yer gonna need it). UPDATE: Oh yeah, i should mention that the SETI project does not need to use threads at all ... you simply download and run the client ... the client will contact the 'master' when it has results to report. Just some socket programming. jeffa L-LL-L--L-LL-L--L-LL-L-- -R--R-RR-R--R-RR-R--R-RR B--B--B--B--B--B--B--B-- H---H---H---H---H---H--- (the triplet paradiddle with high-hat)	[reply]
Re: (jeffa) Re: Threading a job between multiple machines by SyN/AcK (Scribe) on Jul 30, 2003 at 17:14 UTC
Aha, well that is cool, I was wondering how SETI worked. So, with threading then, we talked before I believe about the advantages it provided. Say I wanted to thread a job on my machine, does it gain any speed if I'm on a single processor? I believe you told me it wouldn't, but I was thinking in terms of say I was transfering a file, and I made multiple threads to support multiple connections to the file in the hopes of maximizing my bandwidth usage to get the file. Is this reasonable? Or is the speed increase negligable? What about how they do those distributed attacks of an encryption algorithm? You know what I'm talking about here? They get those large groups of people all over then have some sort of controlling program that tells each machine what key to try... sort of like distributed computing... is this done with threads, or message passing like you mentioned? Sorry, I'm a n00b in this area.	[reply]
(jeffa) 3Re: Threading a job between multiple machines by jeffa (Bishop) on Jul 30, 2003 at 17:39 UTC
IO bound problems benefit from threads on single proc machines. The idea is that if you are waiting on IO, but have other stuff you can be doing in the meantime, then you should be able to spawn a thread to handle the waiting and see 'speed up'. If this is what you want, then i recommend POE (which uses cooperative multitasking ... not threads). As for your second question ... again, no threads are needed. The clients may run 'in the background' via threads, but UNIX systems have these things called processes which are basically 'heavyweight threads' (threads are really just 'lightweight processes'). The client runs 'in the background' and simply connects to the server on some port when it needs to convey information. This could be construed as 'message passing', and it is, but this really is not the same as message passing on something like a Beowulf Cluster. Since you are a graduate student, is there perhaps a parallel processing class you can sign up for? You really should get some 'classic training' on this stuff if it is readily available to you. jeffa L-LL-L--L-LL-L--L-LL-L-- -R--R-RR-R--R-RR-R--R-RR B--B--B--B--B--B--B--B-- H---H---H---H---H---H--- (the triplet paradiddle with high-hat)	[reply]
Re: Threading a job between multiple machines by broquaint (Abbot) on Jul 30, 2003 at 15:34 UTC
I'm afraid there doesn't currently exist a distributed version of perl (at least to the best of my knowledge). Although there's nothing to say that this won't be eventually possible with parrot and a liberal use of continuations ;) But seriously, if anyone were to put in a concerted effort to create a distributed version of perl, then it would probably wiser/easier to target parrot. Update - or you could just listen to jeffa :) HTH `_________ broquaint`	[reply]