Unexpected OO accessor benchmarks

cLive ;-) has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Unexpected OO accessor benchmarks by stvn (Monsignor) on Feb 09, 2007 at 20:08 UTC
Am I missing something, or is that just plain weird? Yes, you are. Try adding this to your script `use Data::Dumper; print Dumper { normal => $self->a(), optimized => $self->b(), direct => $self->c(), };` [download] and you will see this: `$VAR1 = { 'direct' => 123, 'normal' => 123, 'optimized' => 123 };` [download] Essentially you are benchmarking integers :) -stvn	[reply] [d/l] [select]
Re^2: Unexpected OO accessor benchmarks by Fletch (Bishop) on Feb 09, 2007 at 20:10 UTC
Yup, what he should have done is pass an anonymous coderef instead: `normal => sub { $self->a() },` [download] Which gives reasonable results: `Rate normal optimized direct normal 662252/s -- -15% -56% optimized 775194/s 17% -- -49% direct 1515152/s 129% 95% --` [download]	[reply] [d/l] [select]
Re^3: Unexpected OO accessor benchmarks by cLive ;-) (Prior) on Feb 09, 2007 at 20:23 UTC
Thank you - I feel much better now. Oh the fun of pre-processing... `Rate normal optimized direct normal 172740/s -- -28% -60% optimized 239252/s 39% -- -45% direct 433898/s 151% 81% --` [download] That looks a little more like what I expected.	[reply] [d/l]
Re^4: Unexpected OO accessor benchmarks by Fletch (Bishop) on Feb 09, 2007 at 20:42 UTC
Re^5: Unexpected OO accessor benchmarks by cLive ;-) (Prior) on Feb 10, 2007 at 00:55 UTC
Some notes below your chosen depth have not been shown here
Re: Unexpected OO accessor benchmarks by chromatic (Archbishop) on Feb 09, 2007 at 20:35 UTC
Just for fun, I rearranged the order of declaration, putting `get_value()` and `get_value2()` ahead of `a()`, `b()`, and `c()` (as in No More Meaningless Benchmarks!). Here are the results of the first run: `$ perl oo_benchmark.pl (warning: too few iterations for a reliable count) (warning: too few iterations for a reliable count) (warning: too few iterations for a reliable count) Rate normal optimized direct normal 142857143/s -- -14% -57% optimized 166666667/s 17% -- -50% direct 333333333/s 133% 100% --` [download] Here are the results of my modified program: `$ perl oo_benchmark.pl (warning: too few iterations for a reliable count) (warning: too few iterations for a reliable count) (warning: too few iterations for a reliable count) Rate direct normal optimized direct 125000000/s -- -25% -25% normal 166666667/s 33% -- 0% optimized 166666667/s 33% 0% --` [download] I conclude that these kinds of benchmarks are meaningless.	[reply] [d/l] [select]
Re^2: Unexpected OO accessor benchmarks by eric256 (Parson) on Feb 10, 2007 at 02:24 UTC
Actually they arn't meaningless. The fact that I have to do 1_000_000 iterations before I can even tell the difference between the different access method shows me that this isn't a good place to try and improve performance. Unless of course you are doing millions and millions and millions of these accesses every second, then you might care, but i'm betting if thats the case you'd better have some beefy hardware anyways ;) ___________ Eric Hodges	[reply]
Re^2: Unexpected OO accessor benchmarks by cLive ;-) (Prior) on Feb 10, 2007 at 01:08 UTC
I read your node - interesting, thanks. But I can't get results like yours. Could it be because the number of iterations in your test are too low? Putting get_value() and get_value2() declarations at the beginning of program after the "my $self" line, I get: `Rate normal optimized direct normal 174625/s -- -27% -60% optimized 240602/s 38% -- -45% direct 435374/s 149% 81% --` [download] Moving the subs to just above sub a() declaration, I get `Rate normal optimized direct normal 172275/s -- -28% -61% optimized 239252/s 39% -- -46% direct 444444/s 158% 86% --` [download] I've tried a few more re-arrangements of the code layout, and I can't get the reversal that you did. All re-arrangements I've tried have 'direct' at 145-155%. Any chance you can post the code so I can duplicate the results? Logically, I would think that asking for the value of an arrayref element is going to be faster than calling a sub to do so, so the benchmarks appear to be logical to me.	[reply] [d/l] [select]
Re^3: Unexpected OO accessor benchmarks by chromatic (Archbishop) on Feb 10, 2007 at 02:17 UTC
I revised the code slightly: `use Benchmark 'cmpthese'; sub get_value { my $self = shift; return $self->[0]; } sub get_value2 { $_[0]->[0]; } my $self = bless [123], '::main'; cmpthese( 16000000, { normal => sub { $self->get_value() }, optimized => sub { $self->get_value2() }, direct => sub { $self->[0] }, }); exit(0); $ perl oo_benchmark.pl Rate normal optimized direct normal 1462523/s -- -21% -90% optimized 1858304/s 27% -- -88% direct 15238095/s 942% 720% --` [download] Yes, it took 16 million iterations to get an accurate reading on the direct attribute access. With a variant of your original benchmark modified to avoid the double-dispatch penalty, I get: `use Benchmark 'cmpthese'; my $self = bless [123], '::main'; cmpthese( 16000000, { normal => sub { $self->get_value() }, optimized => sub { $self->get_value2() }, direct => sub { $self->[0] }, }); exit(0); sub get_value { my $self = shift; return $self->[0]; } sub get_value2 { $_[0]->[0]; } $ perl oo_benchmark.pl Rate normal optimized direct normal 1725998/s -- -14% -94% optimized 2007528/s 16% -- -93% direct 29629630/s 1617% 1376% --` [download] That represents speedups of around 20%, 11%, and 100%, respectively. I can make the case for a 10% speedup perhaps, but the others are way outside anything I can explain. Thus I'm not sure this benchmark is useful.	[reply] [d/l] [select]