faster way for multiple foreach loops

michbach has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: faster way for multiple foreach loops by ikegami (Patriarch) on Jan 03, 2009 at 13:58 UTC
I don't know. It returns the following really fast for me Global symbol "@list1" requires explicit package name at 733911.pl lin +e 5. Global symbol "$count1" requires explicit package name at 733911.pl li +ne 9. Global symbol "@list1" requires explicit package name at 733911.pl lin +e 9. Global symbol "$num1" requires explicit package name at 733911.pl line + 10. Global symbol "$num2" requires explicit package name at 733911.pl line + 12. Global symbol "$indx" requires explicit package name at 733911.pl line + 14. Global symbol "$count1" requires explicit package name at 733911.pl li +ne 15. Global symbol "$var1" requires explicit package name at 733911.pl line + 17. Global symbol "$num1" requires explicit package name at 733911.pl line + 17. Global symbol "@list1" requires explicit package name at 733911.pl lin +e 17. Global symbol "$indx" requires explicit package name at 733911.pl line + 17. Global symbol "$indx" requires explicit package name at 733911.pl line + 18. Global symbol "$var2" requires explicit package name at 733911.pl line + 19. Global symbol "$num2" requires explicit package name at 733911.pl line + 19. Global symbol "@list1" requires explicit package name at 733911.pl lin +e 19. Global symbol "$indx" requires explicit package name at 733911.pl line + 19. Global symbol "$indx" requires explicit package name at 733911.pl line + 20. Global symbol "$var3" requires explicit package name at 733911.pl line + 21. Global symbol "@list1" requires explicit package name at 733911.pl lin +e 21. Global symbol "$indx" requires explicit package name at 733911.pl line + 21. Global symbol "$indx" requires explicit package name at 733911.pl line + 22. Execution of 733911.pl aborted due to compilation errors. [download] It'll also complete much sooner if it wasn't an infinite loop. What's with the `while ()`? Moving on, I'm not sure we can optimize your code if you don't show it. It's "# further code do anything" that gets executed 181,503 times. It's "# further code do anything" that's going to affect the performance. You might get a tiny gain by rearranging your loops `for ( [ test1 => 10, 22 ], [ test2 => 31, 17 ], [ test3 => 4, 45 ], ) { my ($test, $base1, $base2) = @_; for my $num1 (200..400) { my $var1 = $base1 + $num1; for $num2 (300..600) { my $var2 = $base2 + $num2; ... } } }` [download] Any maybe another tiny gain by eliminating some arithmetic `for ( [ test1 => 10, 22 ], [ test2 => 31, 17 ], [ test3 => 4, 45 ], ) { my ($test, $base1, $base2) = @_; my $var1 = $base1 + 200; for (200..400) { my $var2 = $base2 + 300; for (300..600) { ... ++$var2; } ++$var1; } }` [download] Update: Fixed bug in second snippet.	[reply] [d/l] [select]
Re: faster way for multiple foreach loops by MidLifeXis (Monsignor) on Jan 03, 2009 at 14:05 UTC
Update 2: The OP changed the inner while loop. This node is bunk. What do you consider a "loop". If you are considering the outer while() to be a loop, then I would expect it to take a little while. You are iterating `scalar(@list)301201` or over 500K times for each while. The logic baffles me, however. Your inner loop is iterating over all of the items in `@list1`, and even going well beyond the end of the list, but it looks to me like you only want to loop over them with a step value of 3. Perhaps a loop like: `$indx = 0; while ($indx < $count1) { $var1 = $num1 + $list1[$indx++]; $var2 = $num2 + $list1[$indx++]; $var3 = $list1[$indx++]; # further code... }` [download] Please note the difference between `$list1[$indx...]` and `@list1[$indx...]`. The first (my code) accesses a list element. The second (yours), if you use strict and warnings, should complain to you, and tell you to use the `$list[...]` form (that is, after you declare your variables with `my`). Update: I just did some testing, and your code, 100 times, executes in 41.6 seconds. Mine (yes, it is doing something different, although I suspect, correct) executes in 8.3 seconds. I would also concur with the above poster that the `Further code` will make a huge difference. Update2: To the original poster, please DO NOT update your original node without marking it as such, especially code sections. Your inner loop was originally `foreach(1..$count1) {...}`. --MidLifeXis	[reply] [d/l] [select]
Re^2: faster way for multiple foreach loops by ikegami (Patriarch) on Jan 03, 2009 at 14:09 UTC
Your inner loop is iterating over all of the items in @list1, and even going well beyond the end of the list, No it doesn't. It reads @list1 three at a time from start to end over and over again.	[reply]
Re^3: faster way for multiple foreach loops by MidLifeXis (Monsignor) on Jan 03, 2009 at 14:15 UTC
OP Changed code. --MidLifeXis	[reply]
Re: faster way for multiple foreach loops by graff (Chancellor) on Jan 03, 2009 at 18:53 UTC
As suggested (but not explained) by ikegami in the first reply: whenever you need to do a set of nested loops, the general rule is that it's slightly more efficient (other things being equal) to arrange for the outer-most loop to have the fewest iterations, and the inner-most loop to have the most. Apart from that, you want no more than the absolute minimum set of steps necessary to happen within the inner-most loop -- move as many things to the outer-most loop (or as far outward from the inner-most loop) as possible. And that's about all you can do, without reducing the number of layers or iterations.	[reply]
Re: faster way for multiple foreach loops by bcrowell2 (Friar) on Jan 03, 2009 at 20:38 UTC
Others have suggested moving the foreach(1..3) to the outside, which probably would help a little, although it might not be possible given the details of what your code is actually doing in the "further code" section. If your real application is always going to have exactly 9 elements in @list1, a different approach to optimization might be to unroll the loop like this: `my ($a,$b,$c,$d,$e,$f,$g,$h,$i) = @list1; $var1 = $num1 + $a; $var2 = $num2 + $b; $var3 = $c; #... $var1 = $num1 + $d; $var2 = $num2 + $e; $var3 = $f; #... #etc` [download] But what in the world is the "further code" section doing that takes less time than the loop overhead?? I'm having a hard time imagining that.	[reply] [d/l]
Re: faster way for multiple foreach loops by spx2 (Deacon) on Jan 04, 2009 at 16:08 UTC
if you have: for(d1s - d1e) for(d2s - d2e) for(d3s - d3e) ... for(dns - dne) that means you actually execute the whole thing M=(d1e - d1s)(d2e - d2s)(d3e - d3s)...(dne - dns) times So that means you can make a single loop 1..M and now you're left with the problem of extracting from the counter all the n counters you had initially. If all the intervals you considered initially were equal(let's say the interval was I long) you could just write the new counter in base I and you get the initial indexes,but otherwise...it would probably be a bit more complicated. You probably should convince yourself if the overhead in additional calculations is worth the effort. Good luck :)	[reply]