in reply to Re^9: Parallel::ForkManager is time consuming...takes too long
in thread Parallel::ForkManager is time consuming...takes too long

I just ran the above code with arrays A & B containing 100,000 values each. It completed in a little over half a second

C:\test>920533 >920533.obig
Took 0.609000 seconds

C:\test>wc -l 920533.obig
 100000 920533.obig

[download]

What made you think you needed to use multiprocessing to meet your 30 second target time?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^11: Parallel::ForkManager is time consuming...takes too long
by esolkc (Acolyte) on Aug 16, 2011 at 23:22 UTC
    I have tonnes of such subroutines and the entire code takes 1 minute and 35 seconds. So when I multiprocess all of them it will reduce the time.
[reply]
      I have tonnes of such subroutines and the entire code takes 1 minute and 35 seconds.

      Hm. 95 / 0.609 implies that "tonnes" == 156 subroutines?

      If that is really the case, then recoding them all similar to this:

      use List::Util qw[ sum ];
...
sub subroutine {
    our( @a, @b, @c, @d, @e );
    ( *a, *b, *c, *d, *e ) = @_;

    foreach my $i ( 10 .. $#a ) {
        $c[$i] = $d[$i] = $e[$i] = 0;
        my $b = $b[$i];
        if( ( $b >= 5 ) && ( $b < 7 ) ) {
            $c[$i] += sum @a[ $i-$b+1 .. $i ];
            $c[$i] /= $b;
        }
        elsif( ( $b >= 7 ) && ( $b < 9 ) ) {
            $d[$i] += sum @a[ $i-$b+1 .. $i ];
            $d[$i] /= $b;
        }
        elsif( ( $b >= 9 ) && ( $b < 15 ) ) {
            $e[$i] += sum @a[ $i-$b+1 .. $i ];
            $e[$i] /= $b;
        }
    }
    return;
}

## called like this
my @a = ...;
my @b = ...;
my( @c, @d, @e );
subroutine( \@a, \@b, \@c, \@d, \@e );

      [download]

      which runs in 1/3rd the time of your original, should get you close to your target. If not, then you'd have to sort out the inefficiencies in the rest of your code as well.

      If it is really going to be necessary to multi-process this to achieve your target, then you should not be trying to inject the forks or threads into the loops within these subroutines -- as in your OP code-- but rather call the subroutines themselves concurrently. To advise you further on doing that, I'd need to see the calling code in its entirety.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
        Some of the checks are redundant. This 
                if( ( $b >= 5 ) && ( $b < 7 ) ) {
            $c[$i] += sum @a[ $i-$b+1 .. $i ];
            $c[$i] /= $b;
        }
        elsif( ( $b >= 7 ) && ( $b < 9 ) ) {
            $d[$i] += sum @a[ $i-$b+1 .. $i ];
            $d[$i] /= $b;
        }
        elsif( ( $b >= 9 ) && ( $b < 15 ) ) {
            $e[$i] += sum @a[ $i-$b+1 .. $i ];
            $e[$i] /= $b;
        }

        [download]
        can be reduced to 
                if( $b < 5 ) {
        }
        elsif( $b < 7 ) {
            ( $c[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b;
        }
        elsif( $b < 9 ) {
            ( $d[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b;
        }
        elsif( $b < 15 ) {
            ( $e[$i] += sum @a[ $i-$b+1 .. $i ] ) /= $b;
        }

        [download]
[reply]
[d/l]
[select]
        thank you! I will implement the suggestions. I am sure that will improve the speed. Last question: Is this necessary my $b = $b[$i];?
[reply]
[d/l]

In Section Seekers of Perl Wisdom