Perhaps you need to look up the word "significant". "Can be detected by careful measurement against an isolated subset" doesn't reach the point of "likely to be noticed" which is a minimum requirement for "significant".

Making your whole program run 10% faster isn't really significant as most people won't notice that something took 0.9 seconds instead of a full second. But making one tiny subroutine 10% faster is going to have much less than 10% impact on the speed of any program. Which is why it is very clearly insignificant. To have significance it has to have a noticeable impact. And impact is only noticed in the run time of real code.

I note that your benchmark didn't even use the example code being discussed. I bet the benchmark numbers are even more obviously not significant in that case. How insignificant it looks in that case is just a fraction of how insignificant it will be in real code.

When I run tobyink's benchmark under 5.18, I get a 20% reduction in runtime:

C:\test>\perl5.18\bin\perl.exe
use strict;
use warnings;
use Benchmark qw(cmpthese);

my $name = 'foo';
my $path = 'foobar';
my $sub1 = sub { $_[0]{$path}{$name} = $_[1] if @_ == 2; $_[0]{$path}{
+$name} };
my $sub2 = eval qq[ sub { \$_[0]{$path}{$name} = \$_[1] if \@_ == 2; \
+$_[0]{$path}{$name} } ]
        or die($@);

my $self = {};

cmpthese -1, {
        closure => sub {
                $self->$sub1(0);
                $self->$sub1( $self->$sub1 + $_ ) for 0..10_000;
        },
        stringy => sub {
                $self->$sub2(0);
                $self->$sub2( $self->$sub2 + $_ ) for 0..10_000;
        },
};
^Z
          Rate closure stringy
closure 65.7/s      --    -17%
stringy 79.1/s     20%      --
[download]

If you got a 20% rise, you'd consider it "significant".

If you were offered a 20% reduction in the cost of your next car, you'd consider it "significant".

If you were offered a 20% reduction in your likelihood of early mortality, you'd consider it "significant".

If you were gifted a 20% reduction in your daily processing, such that it reduced it from 25 hours to 20, you'd consider it "significant".

Stop being an ass.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

I would notice getting paid 20% more. I would notice paying 20% less for a car. I might notice if a script ran 20% faster. That's about the minimum percentage reduction in run-time that I'm likely to notice.

But, of course, this isn't even close to a 20% reduction in the run time of a script.

A much more apt comparison would be if you got a 20% discount on... the ashtrays in your car (only after subtracting out "overhead" so you really only got a 10% discount on the actual price of the ashtrays).

This reminds me of a cartoon from my college newspaper that showed a sign saying "50% off books! 50% off supplies! Total savings: 100%!!". You have to believe in magical math like that in order to find a 20% reduction in something that is a tiny part of your total run-time to be "significant".

Take a useful script and have somebody run it once with and once without this optimization. They won't notice that it ran faster. "Not noticeable" is well below "significant".

- tye