The purpose of the quiz was not to claim the superiority of any one given snippet over another. Only in the context that it is presented. We should question the methodology of others, particularly when they start making claims about efficiency and performance.

Take for example the classic example of the bubble sort (which BTW, was mentioned in a reply to a SOPW recently). To say that it is inefficient is a generalization. It can be more efficient than almost any other sort given the right conditions. But it would tick me off if responder had made the generalization and didn't qualify it (they did qualify it). This doesn't happen frequently enough.

As for questions 9 & 10, yes they can be misleading (and that was part of the point) but only if you don't examine it closely or over-generalize. Also, if the results were all predictable, it wouldn't make for a very interesting meditation.

Any differences in your benchmarks whether seemingly major or minor can have a big impact on the metrics. I realize that you already know this, because you knew enough to question it. What worries me is all the people who see metrics and just blindly accept them.

The program used for question 9 was a simple cmpthese:

use strict;
use Benchmark qw(cmpthese);

cmpthese (-30, {
   a => 'foreach (1..$n){$m = $_}',
   b => '$m = $_ for (1..$n)',
   c => 'for ($i=1;$i<$n+1;$i++) {$m = $i}'
});
[download]

No, I didn't go to the extremes I did in question 1 (see my reply to blakem above). For question 10, I did however and wrote one program for each snippet. I also ran them for $n=10,000,000 several times and got the same results:

use strict;
use Benchmark;
my ($i, $m);
my $t0 = new Benchmark;
$_ = "123:45678";
my $n = 5000000;

# snippet goes here.

my $t1 = new Benchmark;
my $td = timediff($t1, $t0);
print "x: ",timestr($td),"\n";
[download]

BTW: thanks for adding the reference to turnsteps tutorial. I read it before starting with Benchmark.pm and it does cover the basics (which I intentionally left out due to the length of my original post, a poor decision in retrospect). Another interesting read is How to Lie with Statistics by Darrell Huff, illustrated by Irving Geis. Not that I lied here of course :)

--Jim