Re: Runing "regular" code with threaded perl

The problem is not with threading. If your program is written to use threads, you won't be able to run it on a version Perl without threads. If your program is not written to use threads, Perl won't use threads and all the concurrency problems you have come from other sources.

It seems to me that your program is launching multiple external programs but is not properly synchronizing them. Without seeing the relevant code, it's hard to tell where your program goes wrong. I recommend looking at Parallell::ForkManager or at the simple runN by Dominus. Both approaches are discussed in Parallelization of heterogenous (runs itself Fortran executables) code.

If you want/need to roll your own parallelisation, I recommend having one or more "queues" into which you put the jobs. Your master program then launches the subprograms to process the jobs in the queues and hopefully has simple enough logic to determine when a job in a queue further down below can be started.

Comment on Re: Runing "regular" code with threaded perl Download Code

Replies are listed 'Best First'.
Re^2: Runing "regular" code with threaded perl by kingskot (Sexton) on Jul 10, 2008 at 07:09 UTC
"If your program is not written to use threads, Perl won't use threads and all the concurrency problems you have come from other sources." What does this mean? Indeed, my program "was not written to use threads" in that is was developed and tested on a single processor machine. Yet the same code fails on a multi-processor machine with threading enabled. The actual code is simple: I make a system call to a routine that outputs a file, then reads the resulting file. The perl script is trying to access the file before the called routine is finished making it (if I load the code in a debugger and step through line by line it runs just fine). How does this translate to "perl not using threads"? Are you suggesting that it's a problem with the OS managing threads?	[reply]
Re^3: Runing "regular" code with threaded perl by pc88mxer (Vicar) on Jul 10, 2008 at 07:14 UTC
What Corion means by "was written to use threads" is simply: does your script contain the statement: `use threads;` [download] If not, the threading features of perl will not be used.	[reply] [d/l]