esolkc has asked for the wisdom of the Perl Monks concerning the following question:

Just to simplfy my problem. I have to codes A and B. Code A is much slower than code B, although I should expect the opposite. I am using a i7 2600K CPU. Any comments are appreciated. Thank you!!

Code A
#!/usr/bin/perl use Parallel::ForkManager; @all_strings=('110110101000101111', '110010101001011101', '110110111001001110', '110011101001011100', '111010101001011100'); my $pm = new Parallel::ForkManager(4); foreach $string (@all_strings) { my $pid = $pm->start and next; print($string,"\n"); $pm->finish; } $pm->wait_all_children;
Code B
#!/usr/bin/perl @all_strings=('110110101000101111', '110010101001011101', '110110111001001110', '110011101001011100', '111010101001011100'); foreach $string (@all_strings) { print($string,"\n"); }

Replies are listed 'Best First'.
Re: Parallel::ForkManager is time consuming...takes too long
by runrig (Abbot) on Aug 12, 2011 at 18:25 UTC
    If you do something non-trivial, fork can be a win. When you do something as trivial as a single print, the overhead makes fork a lose.
      Thank you for you reply, Well, I simplified. For example, replace print($string,"\n"); with ($ouput1, $output2, $output3 ) = &some_subroutine($string); whereas some_subroutine is definded as follows:
      sub some_subroutine { my ($string) = @_; my ($output1, $output2, $output3); if ( $string eq $some_other_string_a ) { $output1 = 1350; $output2 = 9234; $output3 = 8889; } if ( $string eq $some_other_string_b ) { $output1 = 1345; $output2 = 2234; $output3 = 3689; } if ( $string eq $some_other_string_c ) { $output1 = 2256; $output2 = 3370; $output3 = 1340; } @retval = ($output1,$output2,$output3); }
      so the code looks like
      #!/usr/bin/perl use Parallel::ForkManager; @all_strings=('110110101000101111', '110010101001011101', '110110111001001110', '110011101001011100', '111010101001011100'); my $pm = new Parallel::ForkManager(4); foreach $string (@all_strings) { my $pid = $pm->start and next; ($ouput1, $output2, $output3 ) = some_subroutine($string); $pm->finish; } $pm->wait_all_children;
      Or do you have an simple example that really works?
        Still too trivial. Do something that takes some noticeable amount of time. The simplest example would be, e.g., sleep for a few seconds. I often use Parallel::ForkManager for things that individually take a few minutes, it's not much good for things that take a tiny fraction of a second.

        $string == $some_other_string_a

        btw, this is buggy. One needs to use eq to compare strings.

        Running a hundred Perl ops and doing a number of system calls including forking a process in order to avoid running a dozen Perl ops in series is indeed a losing proposition.
Re: Parallel::ForkManager is time consuming...takes too long
by BrowserUk (Patriarch) on Aug 12, 2011 at 23:42 UTC

    You are wasting your time trying speed this up using multi-processing when a simple, single-threaded piece of Perl can process your 100,000 lookups far faster than your 30 second target.

    Less than 2/10ths of a second in fact:

    #! perl -slw use strict; use Time::HiRes qw[ time ]; our $N //= 1e5; my @strings = map sprintf( '%18.18b', int rand( 2**18 ) ), 1 .. $N; my %lookup = ( '101010101010101010' => [ 1350, 9234, 8889 ], '010101010101010101' => [ 1345, 2234, 3689 ], '111111111000000000' => [ 2256, 3370, 1340 ], ); my $start = time; my @vals = @{ $lookup{ $_ } // [0,0,0] } for @strings; printf "$N lookups took %f seconds\n", time() - $start; __END__ C:\test>junk 100000 lookups took 0.137553 seconds

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      This example you have provided is indeed much faster than we had implented. I like to use the same style to solve a similar issue. I have two integer arrays that will be used in a sub-routine within if{} expressions. Each if{}-expression returns some values, as the afoementioned example. I assume I can not use the => operator for the hash in this case.

        Post some code, or pseudo code to explain what you mean, because I cannot makes sense of your description.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.