Re^2: Is fork() my answer to using dual-cpu?

This is exactly what I would suggest, with the added comment that I have used this approach with excellent results.

use Parallel::ForkManager;
$pm = Parallel::ForkManager->new(2);

foreach my $input_file (@files) {
    my $pid = $pm->start and next; 

    system("shell_script < $input_file");

    $pm->finish; # Terminates the child process
}
[download]

Your actual code may vary, depending on requirements

This works well on both Windows and UNIX, and as long as the OS manages the SMP tasks correctly, your threads will use both CPUs (one thread will only use one CPU, obviously) if needed.

Updates:

2005-10.Oct-26 : updated code to change exec() to system(). Thanks to axl163 for pointing that out!

<-radiant.matrix->
A collection of thoughts and links from the minds of geeks
The Code that can be seen is not the true Code
"In any sufficiently large group of people, most are idiots" - Kaa's Law

Comment on Re^2: Is fork() my answer to using dual-cpu? Select or Download Code

Replies are listed 'Best First'.
Re^3: Is fork() my answer to using dual-cpu? by idsfa (Vicar) on Oct 25, 2005 at 19:47 UTC
Actually, you probably want: `$pm = Parallel::ForkManager->new(3);` [download] unless your script has absolutely no disk I/O. Otherwise you will find that you have a processor sitting idle waiting on disk I/O. You might see an improvement with numbers greater than # CPUs + 1, but only in systems with highly parallelized I/O chains. The intelligent reader will judge for himself. Without examining the facts fully and fairly, there is no way of knowing whether vox populi is really vox dei, or merely vox asinorum. — Cyrus H. Gordon	[reply] [d/l]
Re^4: Is fork() my answer to using dual-cpu? by radiantmatrix (Parson) on Oct 26, 2005 at 18:58 UTC
Two things: first, the OP asked specifically about 2 processes; you are absolutely correct that more than 2 is probably better, but the exact number should be arrived at through careful profiling and testing. Second, it is an excellent point that multithreading may be premature optimization: if one's execution speed is bound by some resource other than processor time (e.g. I/O, RAM, network throughput, etc.), then one might actually slow things down by multithreading. Again, profiling and testing are important. Thank you for pointing these out, as I'd clearly forgotten to do so. <-radiant.matrix-> A collection of thoughts and links from the minds of geeks The Code that can be seen is not the true Code "In any sufficiently large group of people, most are idiots" - Kaa's Law	[reply]