in reply to Re^10: Parallel::ForkManager is time consuming...takes too long
in thread Parallel::ForkManager is time consuming...takes too long

I have tonnes of such subroutines and the entire code takes 1 minute and 35 seconds. So when I multiprocess all of them it will reduce the time.
  • Comment on Re^11: Parallel::ForkManager is time consuming...takes too long

Replies are listed 'Best First'.
Re^12: Parallel::ForkManager is time consuming...takes too long
by BrowserUk (Patriarch) on Aug 17, 2011 at 02:35 UTC
    I have tonnes of such subroutines and the entire code takes 1 minute and 35 seconds.

    Hm. 95 / 0.609 implies that "tonnes" == 156 subroutines?

    If that is really the case, then recoding them all similar to this:

    use List::Util qw[ sum ]; ... sub subroutine { our( @a, @b, @c, @d, @e ); ( *a, *b, *c, *d, *e ) = @_; foreach my $i ( 10 .. $#a ) { $c[$i] = $d[$i] = $e[$i] = 0; my $b = $b[$i]; if( ( $b >= 5 ) && ( $b < 7 ) ) { $c[$i] += sum @a[ $i-$b+1 .. $i ]; $c[$i] /= $b; } elsif( ( $b >= 7 ) && ( $b < 9 ) ) { $d[$i] += sum @a[ $i-$b+1 .. $i ]; $d[$i] /= $b; } elsif( ( $b >= 9 ) && ( $b < 15 ) ) { $e[$i] += sum @a[ $i-$b+1 .. $i ]; $e[$i] /= $b; } } return; } ## called like this my @a = ...; my @b = ...; my( @c, @d, @e ); subroutine( \@a, \@b, \@c, \@d, \@e );

    which runs in 1/3rd the time of your original, should get you close to your target. If not, then you'd have to sort out the inefficiencies in the rest of your code as well.

    If it is really going to be necessary to multi-process this to achieve your target, then you should not be trying to inject the forks or threads into the loops within these subroutines -- as in your OP code-- but rather call the subroutines themselves concurrently. To advise you further on doing that, I'd need to see the calling code in its entirety.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Some of the checks are redundant. This
      if( ( $b >= 5 ) && ( $b < 7 ) ) { $c[$i] += sum @a[ $i-$b+1 .. $i ]; $c[$i] /= $b; } elsif( ( $b >= 7 ) && ( $b < 9 ) ) { $d[$i] += sum @a[ $i-$b+1 .. $i ]; $d[$i] /= $b; } elsif( ( $b >= 9 ) && ( $b < 15 ) ) { $e[$i] += sum @a[ $i-$b+1 .. $i ]; $e[$i] /= $b; }
      can be reduced to
      if( $b < 5 ) { } elsif( $b < 7 ) { ( $c[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b; } elsif( $b < 9 ) { ( $d[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b; } elsif( $b < 15 ) { ( $e[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b; }
      thank you! I will implement the suggestions. I am sure that will improve the speed. Last question: Is this necessary my $b = $b[$i];?
        Is this necessary my $b = $b[$i];?

        Strictly necessary for function, no. Quicker and clearer, yes.

        Like a bookmark (or locking a tab) for a page you reference frequently. Perl is not so good as statically compiled languages at sub-expression elimination.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        He added

        <#> gv[*b] s # my $b = $b[$i]; <1> rv2av sKR/1 <0> padsv[$i:2,4] s <2> aelem sK/2 <0> padsv[$b:3,4] sRM*/LVINTRO <2> sassign vKS/2

        in order to replace a large number of

        <#> gv[*b] s # $b[$i] <1> rv2av sKR/1 <0> padsv[$i:2,3] s <2> aelem sK/2

        with

        <0> padsv[$b:3,4] s # $b

        Assuming each of these ops take roughly the same amount of time to execute, this adds up. Basically,

        my $b = $b[$i]; ... $b ... $b ... $b # 6 + 1*3 = 9 ops

        is faster than

        $b[$i] ... $b[$i] ... $b[$i]; # 0 + 4*3 = 12 ops

        Since his code looks up $b[i] 4 to 8 times (depending on the value of $b[$i]), this saves 6 to 18 ops. Per loop pass!