Re: &1 is no faster than %2 when checking for oddness. (Careful what you benchmark)
by BrowserUk (Patriarch) on Nov 16, 2006 at 06:36 UTC
|
I have to agree with lidden, your benchmark is flawed.
- The signal to noise ratio between the code being benchmarked and the code being tested is is so low as to obscure the real differences. subroutine calls are very expensive relative to the operations.
- Both your tests are used in a void context and are possibly being optimised away.
If you put the tests into a boolean context, and loop the tests to negate the overhead of calling the subs, you consistantly get the boolean operation coming out faster than the modulo operation. Even for modest loop counts:
Perl> $loop = 1; $counter = 0;
cmpthese -1, {
and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } },
mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } }
};;
Rate mod and
mod 517876/s -- -2%
and 525790/s 2% --
Perl> $loop = 10; $counter = 0;
cmpthese -1, {
and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } },
mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } }
};;
Rate mod and
mod 229555/s -- -7%
and 245513/s 7% --
Perl> $loop = 100; $counter = 0;
cmpthese -1, {
and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } },
mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } }
};;
Rate mod and
mod 35041/s -- -6%
and 37118/s 6% --
Perl> $loop = 1000; $counter = 0;
cmpthese -1, {
and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } },
mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } }
};;
Rate mod and
mod 3641/s -- -7%
and 3931/s 8% --
Perl> $loop = 1000; $counter = 0;
cmpthese -1, {
and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } },
mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } }
};;
Rate mod and
mod 3725/s -- -7%
and 3989/s 7% --
It's not huge, but it is consistant.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [d/l] |
|
|
#!/usr/bin/perl
use strict;
use warnings;
use Time::HiRes 'gettimeofday';
my $ITERATIONS = 10_000_000;
my $counter1 = 0;
my $counter2 = 0;
my ($s1, $m1) = gettimeofday;
for (1 .. $ITERATIONS) {$a = ++$counter1 & 1}
my ($s2, $m2) = gettimeofday;
for (1 .. $ITERATIONS) {$a = ++$counter2 % 2}
my ($s3, $m3) = gettimeofday;
my $d1 = $s2 - $s1 + ($m2 - $m1) / 1_000_000;
my $d2 = $s3 - $s2 + ($m3 - $m2) / 1_000_000;
printf "And: %.6f Mod: %.6f\n", $d1, $d2;
__END__
And: 2.954181 Mod: 3.093696
And: 3.320696 Mod: 3.321606
And: 3.096083 Mod: 3.218991
And: 2.977993 Mod: 3.113543
And: 2.954634 Mod: 3.100933
And: 2.836724 Mod: 3.168724
And: 2.880770 Mod: 3.472917
And: 3.117684 Mod: 3.552177
And: 3.668317 Mod: 3.616199
And: 3.126708 Mod: 3.126385
| [reply] [d/l] |
|
|
Of course, the biggest flaw is the use the Benchmark module itself. How can use trust a module that sometimes reports negative times?
I do remember seeing that a long time ago, I've not encountered it with recent versions?
However, the main reason for using it is that it seems to get the math right more often than not. Whereas looking at your math above, I am pretty certain that it is suspect.
- Your iteration count is 10_000_000, but your divisor is 1_000_000.
- Without having analysed it too closely, you appear to have a precedence problem in your delta calculations.
This
my $d2 = $s3 - $s2 + ($m3 - $m2) / 1_000_000;
is parsed as
(my $d2 = (($s3 - $s2) + (($m3 - $m2) / 1000000)));
which means that you are adding 1 millionth of the difference in the milliseconds to the difference in the seconds.
Which is about as close to a random number as I can think of :)
Once you fix those problems up, your benchmark method produces much the same results as mine above. And does so consistantly:
c:\test>junk2 -ITERS=1e6
And: 0.000000300 Mod: 0.000000372
c:\test>junk2 -ITERS=10e6
And: 0.000000302 Mod: 0.000000336
c:\test>junk2 -ITERS=100e6
And: 0.000000299 Mod: 0.000000338
You got me on the one counter issue, though if it makes a difference, perl's math is really bad.
But then your use of the global $a is pretty suspect also. Globals are slower than lexicals, so using them has a significant affect upon the results, but switching to lexicals muddies the waters also:
c:\test>junk2 -ITERS=1e6
And: 0.000000403 Mod: 0.000000347
c:\test>junk2 -ITERS=1e6
And: 0.000000409 Mod: 0.000000325
c:\test>junk2 -ITERS=10e6
And: 0.000000386 Mod: 0.000000339
c:\test>junk2 -ITERS=10e6
And: 0.000000384 Mod: 0.000000338
Besides benchmarking slower, it switches the balance of the performance of the operations! The slowdown may be to do with allocation/deallocation of lexical variables(?), but why the type of variable you assign the result of teh expression to should have such a profound significance is beyond me?
That's why I dodged the issue altogether and used ++$counter <op> and 1;. I added this change, to fixes for the problems described above to produce these results:
c:\test>junk2 -ITERS=1e6
And: 0.000000277 Mod: 0.000000286
c:\test>junk2 -ITERS=1e6
And: 0.000000276 Mod: 0.000000271
c:\test>junk2 -ITERS=1e6
And: 0.000000248 Mod: 0.000000283
c:\test>junk2 -ITERS=10e6
And: 0.000000252 Mod: 0.000000277
c:\test>junk2 -ITERS=10e6
And: 0.000000252 Mod: 0.000000278
c:\test>junk2 -ITERS=10e6
And: 0.000000255 Mod: 0.000000278
c:\test>junk2 -ITERS=100e6
And: 0.000000250 Mod: 0.000000277
Which are a) entirely self consistant; b) are consistant with the results produced by two other benchmarking methods--both my use of Benchmark and lidden's use of an external timer.
The corrected benchmark is:
#! perl -slw
use strict;
use Time::HiRes 'gettimeofday';
our $ITERS ||= 10_000_000;
my $counter1 = 0;
my $counter2 = 0;
my $s1 = gettimeofday;
for (1 .. $ITERS) { ++$counter1 & 1 and 1 }
my $s2 = gettimeofday;
for (1 .. $ITERS) { ++$counter2 % 2 and 1 }
my $s3 = gettimeofday;
my $d1 = ( $s2 - $s1 ) / $ITERS;
my $d2 = ( $s3 - $s2 ) / $ITERS;
printf "And: %.9f Mod: %.9f\n", $d1, $d2;
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [d/l] [select] |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Re: &1 is no faster than %2 when checking for oddness. Oh well.
by brian_d_foy (Abbot) on Nov 16, 2006 at 06:57 UTC
|
When you benchmark something and it happens 4 million times a second, you're doing something wrong. You should look at those rates and wonder why they are so high before thinking about any of the other numbers. Just because Benchmark gives you a nice report doesn't mean it actually means anything. :)
| [reply] |
Re: &1 is no faster than %2 when checking for oddness. Oh well.
by grinder (Bishop) on Nov 16, 2006 at 09:21 UTC
|
no difference between the speed of ++$counter & 1 and ++$counter % 2. I thought there might be but I guess not
When you get down to the metal, an AND instruction probably executes in one clock cycle. A DIV/MOD instruction on a classic CISC architecture might take 40, then again, it might also take one clock cycle too. If the code is balanced on a razor's edge just so, your code that does the AND might just happen to consistently cause an L2 cache miss, and then your modulo will appear to be faster.
I would imagine that even if the machine instruction that performs a divide/modulo was two orders of magnitude slower, from the point of view of the perl interpreter there's so much else going on that the difference is going to be lost in the noise.
Which in fact, it turned out to be :)
• another intruder with the mooring in the heart of the Perl
| [reply] |
Re: &1 is no faster than %2 when checking for oddness. Oh well.
by Tanktalus (Canon) on Nov 16, 2006 at 16:11 UTC
|
About all this entire thread has proven to me is that the difference is not significant, regardless of which one comes out on top. So I'll repeat myself here: choose the behaviour that says what you mean over the one that seems to be faster (today).
As tye pointed out, this type of optimisation can run you into loads of trouble if you forget about the range of your integers. However, if you stick to using % to determine "oddness", you're much more likely to stick within the range that perl supports. Getting it right for sum(1..n) when n gets up over 92_681, for example, may be important later. And because perl abstracts numbers that are too big to be held in an integer, the modulo continues to work.
Use & when you want to say "I'm twiddling bits". Use % when you want to say "I'm finding the remainder (if any)". Since the definition of odd (at least on the set of integers) is "not even" and the definition of even is "divisible by two with no remainder", it seems only logical to use % to find it.
Of course, it's also easily extensible to finding multiples (or not) of other numbers, too... Let the hardware guys worry about these optimisations. At some point, some guy at AMD or Intel or whomever will decide that evenness is important enough that when it seems a modulo operation where the divisor is 2 (or a power of 2), it switches to bit-twiddling. Even then, perl will still get it right for large numbers because it won't use the normal modulo operator when the numbers are too big, taking advantage of the hardware's ability to optimise normal numbers while not losing the accuracy of perl's abstractions.
| [reply] |
|
|
| [reply] [d/l] [select] |
|
|
If perl's abstraction of numbers isn't good enough for you, it's easy enough to fix. use bignum;.
'Oddness' is an inherently integer concept and a boolean condition. Using an inherently non-integer, non-boolean operator to test for an inherently integer, boolean condition is just plain wrong!
This I entirely, 100% agree with. Absolutely. However, I think you're getting a bit paranoid here. In my definition of oddness, I stipulated that it was confined to the set of integers - so talking about floating point concerns seems a bit orthogonal to that question. So, in the range of integers (from -inf to +inf), perl can't quite handle 'em all. But, that said, the concept itself is simple: an integer is odd iff it is not even. An integer is even iff its remainder when divided by two is zero. Thus, the simple boolean statement of $x % 2 != 0 (which is just a minor simplification from the more literal !($x % 2 == 0)) Or, a simpler definition of oddness is an integer which is not evenly divisible by two - which gives us the same boolean test. Then, when dealing with huge numbers, insert the bignum pragma, and you're done. A bit slow, perhaps, but accurate.
As to your comments on bitwise operations on non-integral reals, I think that's just reminiscent of C where some people try to do that and think they're being smart. Instead of testing "if (x < 0)", they try to test the negative bit in the double, wherever that is. (Then they get hurt in a similar way when they somehow get a "negative zero".) Personally, I wouldn't mind a warning from perl's runtime for it - bit twiddling with reals is probably not what you were intending to do.
| [reply] [d/l] [select] |
|
|
|
|
use Math::BigInt;
my $sum = Math::BigInt->new(2);
$sum->bpow(30);
$sum = ($sum * $sum + $sum) / 2;
print $sum;
__DATA__
576460752840294400
This is mostly in jest but it does illustrate that programmer knowledge of the specific task at hand is more valuable then generalizations. There are similar formulas for summing just the odd or even numbers from 1 .. N as well.
| [reply] [d/l] [select] |
|
|
|
|
|
Re: &1 is no faster than %2 when checking for oddness. Oh well.
by bart (Canon) on Nov 16, 2006 at 21:15 UTC
|
You're getting old. A trick like this made a huge difference in the time where division was an expensive operation. But since the FPU coprocessors became common, about 10 years ago, division is, in practice, as fast as any other operation. So it will no longer make much of a difference. Definitely not as much as it did 10 years ago.
Furthermore, you're testing this in Perl, not in C or in assembler. Perl has such a huge overhead that even if bare division took twice as long as bitwise and, by the time you get to the level of interpreted Perl, you lost most of your advantage. | [reply] |
|
|
| [reply] |
Re: &1 is no faster than %2 when checking for oddness. Oh well.
by diotalevi (Canon) on Nov 16, 2006 at 00:04 UTC
|
Someone asked that I run this with counts of -10 and -20. Here's three consecutive runs with the COUNT being -10. So now % appears faster but only just and maybe only at random times. So. I think I should be synching with the phase of the moon and whether some planet or other is ascension or some other astrology thing to decide which is faster at any given moment.
Rate and mod
and 2517343/s -- -13%
mod 2898815/s 15% --
[jbenjore@w3d211 src]$ perl odd.pl
Rate and mod
and 2545188/s -- -9%
mod 2792512/s 10% --
[jbenjore@w3d211 src]$ perl odd.pl
Rate and mod
and 2667452/s -- -2%
mod 2727161/s 2% --
| [reply] [d/l] |
Re: &1 is no faster than %2 when checking for oddness. Oh well.
by lidden (Curate) on Nov 16, 2006 at 05:39 UTC
|
I compared
time perl -wle '$a = $_ & 1 for 1..50_000_000'
with
time perl -wle '$a = $_ % 2 for 1..50_000_000'
The & 1 version is always a little faster. | [reply] [d/l] [select] |
|
|
| [reply] |
|
|
Perl> use Devel::Peek;;
Perl> $n = 9_999_999_998; print Dump( $n ); print $n & 1; print Dump(
+$n );;
SV = NV(0x1844044) at 0x19920c4
REFCNT = 1
FLAGS = (NOK,pNOK)
NV = 9999999998
1
SV = PVNV(0x1985a0c) at 0x19920c4
REFCNT = 1
FLAGS = (NOK,pIOK,pNOK,IsUV)
UV = 4294967295
NV = 9999999998
PV = 0
The silent 'promotion' of the NV to an UV with data loss is very messy.
It's also kinda schizophrenic when you consider that perl does differentiate between integer and real operations some places. Take the arbitrary and intensely annoying case of:
Perl> $c++ for 1 .. 2**32;;
[Range iterator outside integer range at (eval 7) line 1, <STDIN> line
+ 3.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [d/l] [select] |
|
|
|
|
|
Re: &1 is no faster than %2 when checking for oddness. Oh well.
by radiantmatrix (Parson) on Nov 22, 2006 at 17:45 UTC
|
Screw the benchmarks! In fact, screw math!
sub is_odd {
my $int = shift;
if ( $int =~ /\D/ ) { die "Not an integer, dummy!" }
substr($int,-1) =~ /[13579]/;
}
Yes, it's a joke.
<–radiant.matrix–>
Ramblings and references
The Code that can be seen is not the true Code
I haven't found a problem yet that can't be solved by a well-placed trebuchet
| [reply] [d/l] |