Re^3: An efficient way to parallelize loops

Your first point, unluckely is not applicable, due to the fact that depending on the element in @{$categories{$k}->{traces}} matched, the sum below will be applied to a different element in the @lista array.

I don't understand what you're saing. The code you showed us changes neither %categories nor $k. Why should anything change if you do the lookup outside the loop?

my @a = @{$categories{$k}->{traces};
OUTER: while( my $line = <GZIP> ) {
    for ( my $i = 0; $i < @a }; $i++ ) {
        if ( $line =~ /^($a[$i]->{regex})/ ) {
            my @lista = split /;/, $line;
            $A += $lista[$a[$i]->{calc}}[0]];
            $B += $lista[$a[$i]->{calc}}[1]];
            $C += $lista[$a[$i]->{calc}}[2]];
            next OUTER;
        }
    }
}
[download]

This should do exactly the same as your code, only more efficient.

The version with while is at least as fast as the version with for, and uses much less memory.

Still I'd like to emphasize my first point again: Benchmark and profile before starting to optimize (and before even thinking of parallelization).

Comment on Re^3: An efficient way to parallelize loops Select or Download Code

Replies are listed 'Best First'.
Re^4: An efficient way to parallelize loops by Deus Ex (Scribe) on Jun 01, 2010 at 10:18 UTC
Dammit! I misunderstood what you wrote before: i had intended you suggested to move the whole "if" statement outside of the loop, not the lookup for the variables' values in the statement. Sorry! You were right. I'll also do the profiling, to see where the bottlenecks are. Thank you very much again!	[reply]
Re^4: An efficient way to parallelize loops by Deus Ex (Scribe) on Jun 01, 2010 at 13:23 UTC
Do you think that moving outside the loop the split() would be useful? I also thought to do like this: `open(GZIP, "<:gzip", "$path") or die "$!\n"; my @a = @{$categories{$k}->{traces}}; LOOP: while ( my $line = <GZIP>) { my @lista = split /;/, $line; my $head = $lista[0]; for ( my $i = '0'; $i < scalar @a; $i+ ++ ) { if ( $head =~ /^($a[$i]->{rege +x})$/ ) { $A += $lista[$a[$i]->{ +calc}[0]]; $B += $lista[$a[$i]->{ +calc}[1]]; $C += $lista[$a[$i]->{ +calc}[2]]; next LOOP; } } } close(GZIP);` [download] I put the first element of the @lista array as the pattern to test, cause it is there; no need to test the whole line.	[reply] [d/l]