Re^12: Parallel::ForkManager is time consuming...takes too long

I have tonnes of such subroutines and the entire code takes 1 minute and 35 seconds.

Hm. 95 / 0.609 implies that "tonnes" == 156 subroutines?

If that is really the case, then recoding them all similar to this:

use List::Util qw[ sum ];
...
sub subroutine {
    our( @a, @b, @c, @d, @e );
    ( *a, *b, *c, *d, *e ) = @_;

    foreach my $i ( 10 .. $#a ) {
        $c[$i] = $d[$i] = $e[$i] = 0;
        my $b = $b[$i];
        if( ( $b >= 5 ) && ( $b < 7 ) ) {
            $c[$i] += sum @a[ $i-$b+1 .. $i ];
            $c[$i] /= $b;
        }
        elsif( ( $b >= 7 ) && ( $b < 9 ) ) {
            $d[$i] += sum @a[ $i-$b+1 .. $i ];
            $d[$i] /= $b;
        }
        elsif( ( $b >= 9 ) && ( $b < 15 ) ) {
            $e[$i] += sum @a[ $i-$b+1 .. $i ];
            $e[$i] /= $b;
        }
    }
    return;
}

## called like this
my @a = ...;
my @b = ...;
my( @c, @d, @e );
subroutine( \@a, \@b, \@c, \@d, \@e );
[download]

which runs in 1/3rd the time of your original, should get you close to your target. If not, then you'd have to sort out the inefficiencies in the rest of your code as well.

If it is really going to be necessary to multi-process this to achieve your target, then you should not be trying to inject the forks or threads into the loops within these subroutines -- as in your OP code-- but rather call the subroutines themselves concurrently. To advise you further on doing that, I'd need to see the calling code in its entirety.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re^12: Parallel::ForkManager is time consuming...takes too long Download Code

Replies are listed 'Best First'.
Re^13: Parallel::ForkManager is time consuming...takes too long by ikegami (Patriarch) on Aug 17, 2011 at 03:15 UTC
Some of the checks are redundant. This `if( ( $b >= 5 ) && ( $b < 7 ) ) { $c[$i] += sum @a[ $i-$b+1 .. $i ]; $c[$i] /= $b; } elsif( ( $b >= 7 ) && ( $b < 9 ) ) { $d[$i] += sum @a[ $i-$b+1 .. $i ]; $d[$i] /= $b; } elsif( ( $b >= 9 ) && ( $b < 15 ) ) { $e[$i] += sum @a[ $i-$b+1 .. $i ]; $e[$i] /= $b; }` [download] can be reduced to `if( $b < 5 ) { } elsif( $b < 7 ) { ( $c[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b; } elsif( $b < 9 ) { ( $d[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b; } elsif( $b < 15 ) { ( $e[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b; }` [download]	[reply] [d/l] [select]
Re^13: Parallel::ForkManager is time consuming...takes too long by esolkc (Acolyte) on Aug 17, 2011 at 17:38 UTC
thank you! I will implement the suggestions. I am sure that will improve the speed. Last question: Is this necessary `my $b = $b[$i];`?	[reply] [d/l]
Re^14: Parallel::ForkManager is time consuming...takes too long by BrowserUk (Patriarch) on Aug 17, 2011 at 23:23 UTC
Is this necessary `my $b = $b[$i];`? Strictly necessary for function, no. Quicker and clearer, yes. Like a bookmark (or locking a tab) for a page you reference frequently. Perl is not so good as statically compiled languages at sub-expression elimination. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l]
Re^14: Parallel::ForkManager is time consuming...takes too long by ikegami (Patriarch) on Aug 17, 2011 at 23:35 UTC
He added `<#> gv[b] s # my $b = $b[$i]; <1> rv2av sKR/1 <0> padsv[$i:2,4] s <2> aelem sK/2 <0> padsv[$b:3,4] sRM/LVINTRO <2> sassign vKS/2` [download] in order to replace a large number of `<#> gv[b] s # $b[$i] <1> rv2av sKR/1 <0> padsv[$i:2,3] s <2> aelem sK/2` [download] with `<0> padsv[$b:3,4] s # $b` [download] Assuming each of these ops take roughly the same amount of time to execute, this adds up. Basically, `my $b = $b[$i]; ... $b ... $b ... $b # 6 + 13 = 9 ops` [download] is faster than `$b[$i] ... $b[$i] ... $b[$i]; # 0 + 43 = 12 ops` [download] Since his code looks up `$b[i]` 4 to 8 times (depending on the value of `$b[$i]`), this saves 6 to 18 ops. Per loop pass!*	[reply] [d/l] [select]