Re^2: Parallel::ForkManager is time consuming...takes too long

Thank you for you reply, Well, I simplified. For example, replace print($string,"\n"); with ($ouput1, $output2, $output3 ) = &some_subroutine($string); whereas some_subroutine is definded as follows:

sub some_subroutine 
{
        my ($string) = @_;
    my ($output1, $output2, $output3);

    if (  $string eq $some_other_string_a )
    {
           $output1 = 1350;
           $output2 = 9234;
           $output3 = 8889;
    }
    if ( $string eq $some_other_string_b )
    {
           $output1 = 1345;
           $output2 = 2234;
           $output3 = 3689;
    }
    if ( $string eq $some_other_string_c )
    {
           $output1 = 2256;
           $output2 = 3370;
           $output3 = 1340;
    }
    @retval = ($output1,$output2,$output3);
}
[download]

so the code looks like

#!/usr/bin/perl
use Parallel::ForkManager;

@all_strings=('110110101000101111', 
              '110010101001011101', 
              '110110111001001110',
              '110011101001011100',
              '111010101001011100');

my $pm = new Parallel::ForkManager(4);
foreach $string (@all_strings)
{
  my $pid = $pm->start and next;
  ($ouput1, $output2, $output3 ) = some_subroutine($string);
  $pm->finish;
}
$pm->wait_all_children;
[download]

Or do you have an simple example that really works?

Comment on Re^2: Parallel::ForkManager is time consuming...takes too long Select or Download Code

Replies are listed 'Best First'.
Re^3: Parallel::ForkManager is time consuming...takes too long by runrig (Abbot) on Aug 12, 2011 at 18:49 UTC
Still too trivial. Do something that takes some noticeable amount of time. The simplest example would be, e.g., sleep for a few seconds. I often use Parallel::ForkManager for things that individually take a few minutes, it's not much good for things that take a tiny fraction of a second.	[reply]
Re^4: Parallel::ForkManager is time consuming...takes too long by esolkc (Acolyte) on Aug 12, 2011 at 23:26 UTC
Is a double loop over 1,000,000 and 20,000 not big enough for fork. I need the process to be finished in less than 30 sec., while using all cores. The sleep() example is difficult to compare to the issue I outlined. I could not find in the documentation how the forking is implemented.	[reply]
Re^5: Parallel::ForkManager is time consuming...takes too long by runrig (Abbot) on Aug 14, 2011 at 17:04 UTC
I don't know how long your 1,000,000 or 20,000 long loops take, but if you're forking 20 billion times, I bet it's going to be slower. If you can divide your 20 billion loops into some smaller number and only fork that many times (and do more work in each loop), then maybe you can get some benefit from forking.	[reply]
Re^3: Parallel::ForkManager is time consuming...takes too long by ikegami (Patriarch) on Aug 12, 2011 at 19:07 UTC
`$string == $some_other_string_a` btw, this is buggy. One needs to use `eq` to compare strings.	[reply] [d/l] [select]
Re^3: Parallel::ForkManager is time consuming...takes too long by ikegami (Patriarch) on Aug 12, 2011 at 20:49 UTC
Running a hundred Perl ops and doing a number of system calls including forking a process in order to avoid running a dozen Perl ops in series is indeed a losing proposition.	[reply]
Re^4: Parallel::ForkManager is time consuming...takes too long by esolkc (Acolyte) on Aug 12, 2011 at 23:08 UTC
At least I have the syntax right. So but when should I use forking? All examples that I can find somewhere in the Net are as simple as the example I have provide. I have several loops and subroutines which can be distributed on different CPU-cores.	[reply]
Re^5: Parallel::ForkManager is time consuming...takes too long by ikegami (Patriarch) on Aug 13, 2011 at 00:05 UTC
When it helps and the extra complexity is acceptable.	[reply]