But I am pleasantly surprised by the low overhead of loading strict: being able to load strict more than 1 million times per second is nice. But on the other hand, it indicates to me that the OS has the file in RAM already and is serving it from there. So in that sense the benchmark is also flawed ;-).

Anyway, I think we've proven there is not a lot to be gained by not using strict from a performance point of view. And that there is a lot to be lost from a development / maintenance point of view (if you don't use strict).

Liz

But on the other hand, it indicates to me that the OS has the file in RAM already and is serving it from there. So in that sense the benchmark is also flawed ;-).

Yes, and no. If you have a situation where the OS hasn't cached 'strict.pm', it means 'strict.pm' isn't used relatively often. But the less something is used, the less it's a bottleneck, and the less impact higher load times have.

So, you are right, but in situations where you are right, the outcome of the Benchmark isn't that important.

Abigail

This is completely OT as far as this thread is concerned, and is totally meaningless (probably) anyway, but I'd dearly like an explaination anyway.

Reading tachyon's modified benchmark it struck me that if purely the load time was of interest, then the simple expedient of deleting a non-existant hash key probably didn't quite cut it when it came to removing the "administrative overhead" of use from the equation, so I modified it slightly to create and delete a key each time to see the effect that had.

The results are surprising, and somewhat confusing.

Pre-tye code

The biggy!! Post-tye correction.

use Benchmark 'cmpthese';

my %hash = ( key => 'value' );

cmpthese( -3, {
   with =>        sub {
       delete $INC{'strict.pm'};
       require 'strict.pm';
       strict::import( 'refs' );
       my $a = 1;
       my $b = 2
    },
   without =>    sub {
       $hash{key} = delete $hash{key};
       my $a = 1; my $b = 2
    },
} );

__END__
P:\test>..\bin\perl.exe junk.pl8
           Rate    with without
with      137/s      --    -99%
without 24226/s  17576%      --
[download]

Which apart from the fact that tachyon's machine is about 20x quicker than mine, is a mildly interesting result. However, I'm not sure that I'm really comparing eggs with eggs, so I had another go.

This time, I thought I would force Benchmark to re-compile (eval) both snippets each time it exercised them, rather than just call a pre-compiled sub and I got these results.

Pre-tye code

Post-tye corrected benchmark

use Benchmark 'cmpthese';

my %hash = ( key => 'value' );

cmpthese( -10, {
   with =>    q[
       BEGIN{ delete $INC{'strict.pm'}; use strict 'refs' }
       my $a = 1; my $b = 2
    ],
   without => q[
       BEGIN{ $hash{'key'} = delete $hash{key} }
       my $a = 1; my $b = 2
    ],
} );

__END__
P:\test>..\bin\perl.exe junk.pl8
            Rate    with without
with    368780/s      --     -2%
without 375558/s      2%      --

P:\test>..\bin\perl.exe junk.pl8
            Rate    with without
with    371638/s      --     -1%
without 374367/s      1%      --
[download]

Which, without drawing any conclusions as I am still not particularly certain that I am really comparing eggs with eggs, is much closer to my real world experience of benchmarking complete programs with and without use strict.

Whenever I tried this before, I nearly always found that the differences were marginal, if detectable.

Did any of my attempts get closer to a real world test?

$INC{strict.pm}
[download]

That is so ironic that I nearly laughed out loud. Perhaps you should add "use strict" to your benchmark!

                - tye

My only defense Sir, I cannot tell a lie...it was him :)

---

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Hooray!

...machine is about 20x quicker than mine :-)

The machine that ran on was a 1.4GHz Athlon, with IDE disks. Out of interest I ran it on one of the devel servers (Quad PIII Xeon 2GHz, RAID 5 SCSI really fast disks) Of course you only get to use one processor but there was a negligible performance difference.

[root@devel3 root]# perl test.pl
Benchmark: timing 10000000 iterations of with, without...
      with:  9 wallclock secs ( 8.64 usr +  0.00 sys =  8.64 CPU) @ 11
+57407.41/s (n=10000000)
   without:  4 wallclock secs ( 4.05 usr +  0.00 sys =  4.05 CPU) @ 24
+69135.80/s (n=10000000)
             Rate    with without
with    1157407/s      --    -53%
without 2469136/s    113%      --
[download]

What did you run it on out of interest?

On the Benchmark side the BEGIN case have identical (and much faster results) because Perl only executes them once and then effectively does a NOP on that code section. At least that is what I suspect....

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

What did you run it on out of interest?

233MHz P-II.

On the Benchmark side the BEGIN case have identical (and much faster results) because Perl only executes them once and then effectively does a NOP on that code section. At least that is what I suspect....

Given that the BEGIN{}s are wrapped in strings, the won't actually be seen by the compiler until eval time, which would mean that they are re-compiled each time the benchmark is iterated. Unless Benchmark wraps string code arguements in a subroutine wrapper and takes a coderef to the compiled code and invokes it via that for the second and subsequent runs, which is a possibility. I'll try to confirm or deny that.

---

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Hooray!

Which apart from the fact that tachyon's machine is about 20x quicker than mine, is a mildly interesting result.

I guess I fail to see how this is even mildly interesting. The rate of loading on your box appears to be 7,281 loads per second. For something that happens only once in a program, can that even be considered any kind of performance hit? I think not.

One of the reasons load-time can be an issue, is for webservers that don't use mod_perl. Using modules that impose a large load-time hit can cause a measurable degradation if they are being loaded 100's of time a second as with a busy website cgi. That, as I understand it, is the very basis of mod_perls existance.

What made it interesting was the very fact that, if the benchmark is valid, it shows that the runtime hit due to strict is slightly more than 1/10 th of a millisecond, which when compared to beasts like cgi and the various DBI::* modules is so miniscule as to be totally irrelevant, which I found mildly interesting in the context of this thread.

---

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Hooray!