in reply to &1 is no faster than %2 when checking for oddness. Oh well.

I have to agree with lidden, your benchmark is flawed.

  1. The signal to noise ratio between the code being benchmarked and the code being tested is is so low as to obscure the real differences. subroutine calls are very expensive relative to the operations.
  2. Both your tests are used in a void context and are possibly being optimised away.

If you put the tests into a boolean context, and loop the tests to negate the overhead of calling the subs, you consistantly get the boolean operation coming out faster than the modulo operation. Even for modest loop counts: 

Perl> $loop = 1; $counter = 0; 
cmpthese -1, { 
    and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } }, 
    mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } } 
};;
        Rate mod and
mod 517876/s  -- -2%
and 525790/s  2%  --

Perl> $loop = 10; $counter = 0; 
cmpthese -1, { 
    and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } }, 
    mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } } 
};;
        Rate mod and
mod 229555/s  -- -7%
and 245513/s  7%  --

Perl> $loop = 100; $counter = 0; 
cmpthese -1, { 
    and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } }, 
    mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } } 
};;
       Rate mod and
mod 35041/s  -- -6%
and 37118/s  6%  --

Perl> $loop = 1000; $counter = 0; 
cmpthese -1, { 
    and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } }, 
    mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } } 
};;
      Rate mod and
mod 3641/s  -- -7%
and 3931/s  8%  --

Perl> $loop = 1000; $counter = 0; 
cmpthese -1, { 
    and=>sub{ for(1..$loop){ ++$counter & 1 and 1 } }, 
    mod=>sub{ for(1..$loop){ ++$counter % 2 and 1 } } 
};;
      Rate mod and
mod 3725/s  -- -7%
and 3989/s  7%  --

[download]

It's not huge, but it is consistant.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re: &1 is no faster than %2 when checking for oddness. (Careful what you benchmark)
  • Download Code

Replies are listed 'Best First'.
Re^2: &1 is no faster than %2 when checking for oddness. (Careful what you benchmark)
by Anonymous Monk on Nov 16, 2006 at 09:01 UTC
    Your benchmark still has a flaw; it's using the same variable to increment. So one case is incrementing from 0, the other from where the first one left off. It may not make a difference, but how can you be sure.

    Of course, the biggest flaw is the use the Benchmark module itself. How can use trust a module that sometimes reports negative times? 

    #!/usr/bin/perl

use strict;
use warnings;
use Time::HiRes 'gettimeofday';

my $ITERATIONS = 10_000_000;

my $counter1  = 0;
my $counter2  = 0;

my ($s1, $m1) = gettimeofday;
for (1 .. $ITERATIONS) {$a = ++$counter1 & 1}
my ($s2, $m2) = gettimeofday;
for (1 .. $ITERATIONS) {$a = ++$counter2 % 2}
my ($s3, $m3) = gettimeofday;

my $d1 = $s2 - $s1 + ($m2 - $m1) / 1_000_000;
my $d2 = $s3 - $s2 + ($m3 - $m2) / 1_000_000;

printf "And: %.6f  Mod: %.6f\n", $d1, $d2;

__END__
And: 2.954181  Mod: 3.093696
And: 3.320696  Mod: 3.321606
And: 3.096083  Mod: 3.218991
And: 2.977993  Mod: 3.113543
And: 2.954634  Mod: 3.100933
And: 2.836724  Mod: 3.168724
And: 2.880770  Mod: 3.472917
And: 3.117684  Mod: 3.552177
And: 3.668317  Mod: 3.616199
And: 3.126708  Mod: 3.126385

    [download]
[reply]
[d/l]
      Of course, the biggest flaw is the use the Benchmark module itself. How can use trust a module that sometimes reports negative times?

      I do remember seeing that a long time ago, I've not encountered it with recent versions?

      However, the main reason for using it is that it seems to get the math right more often than not. Whereas looking at your math above, I am pretty certain that it is suspect.

      • Your iteration count is 10_000_000, but your divisor is 1_000_000.
      • Without having analysed it too closely, you appear to have a precedence problem in your delta calculations.

        This 

        my $d2 = $s3 - $s2 + ($m3 - $m2) / 1_000_000;

        [download]

        is parsed as 

        (my $d2 = (($s3 - $s2) + (($m3 - $m2) / 1000000)));

        [download]

        which means that you are adding 1 millionth of the difference in the milliseconds to the difference in the seconds.

        Which is about as close to a random number as I can think of :)

      Once you fix those problems up, your benchmark method produces much the same results as mine above. And does so consistantly: 

      c:\test>junk2 -ITERS=1e6
And: 0.000000300  Mod: 0.000000372

c:\test>junk2 -ITERS=10e6
And: 0.000000302  Mod: 0.000000336

c:\test>junk2 -ITERS=100e6
And: 0.000000299  Mod: 0.000000338

      [download]

      You got me on the one counter issue, though if it makes a difference, perl's math is really bad.

      But then your use of the global $a is pretty suspect also. Globals are slower than lexicals, so using them has a significant affect upon the results, but switching to lexicals muddies the waters also: 

      c:\test>junk2 -ITERS=1e6
And: 0.000000403  Mod: 0.000000347

c:\test>junk2 -ITERS=1e6
And: 0.000000409  Mod: 0.000000325

c:\test>junk2 -ITERS=10e6
And: 0.000000386  Mod: 0.000000339

c:\test>junk2 -ITERS=10e6
And: 0.000000384  Mod: 0.000000338

      [download]

      Besides benchmarking slower, it switches the balance of the performance of the operations! The slowdown may be to do with allocation/deallocation of lexical variables(?), but why the type of variable you assign the result of teh expression to should have such a profound significance is beyond me?

      That's why I dodged the issue altogether and used ++$counter <op> and 1;. I added this change, to fixes for the problems described above to produce these results: 

      c:\test>junk2 -ITERS=1e6
And: 0.000000277  Mod: 0.000000286

c:\test>junk2 -ITERS=1e6
And: 0.000000276  Mod: 0.000000271

c:\test>junk2 -ITERS=1e6
And: 0.000000248  Mod: 0.000000283

c:\test>junk2 -ITERS=10e6
And: 0.000000252  Mod: 0.000000277

c:\test>junk2 -ITERS=10e6
And: 0.000000252  Mod: 0.000000278

c:\test>junk2 -ITERS=10e6
And: 0.000000255  Mod: 0.000000278

c:\test>junk2 -ITERS=100e6
And: 0.000000250  Mod: 0.000000277

      [download]

      Which are a) entirely self consistant; b) are consistant with the results produced by two other benchmarking methods--both my use of Benchmark and lidden's use of an external timer.

      The corrected benchmark is: 

      #! perl -slw
use strict;
use Time::HiRes 'gettimeofday';

our $ITERS ||= 10_000_000;

my $counter1  = 0;
my $counter2  = 0;

my $s1 = gettimeofday;
for (1 .. $ITERS) { ++$counter1 & 1 and 1 }
my $s2 = gettimeofday;
for (1 .. $ITERS) { ++$counter2 % 2 and 1 }
my $s3 = gettimeofday;

my $d1 = ( $s2 - $s1 ) / $ITERS;
my $d2 = ( $s3 - $s2 ) / $ITERS;

printf "And: %.9f  Mod: %.9f\n", $d1, $d2;

      [download]
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]
        Your iteration count is 10_000_000, but your divisor is 1_000_000

        Indeed. Why should they be the same? The interator counts iterations. The divisor divides the number of microseconds. I'd divide by a million regardless how many times I iterated.

        This 
        my $d2 = $s3 - $s2 + ($m3 - $m2) / 1_000_000;

        [download]
        is parsed as 
        (my $d2 = (($s3 - $s2) + (($m3 - $m2) / 1000000)));

        [download]
        Indeed it is! Exactly as I intent it! Surprise, surprise.
        which means that you are adding 1 millionth of the difference in the milliseconds to the difference in the seconds.
        Wrong. RTFM. gettimeofday returns the number of microseconds. And guess what? There are a million microseconds in a second. Even in the USA and the UK.

        Now, beside being wrong, you are also inconsistent. In your first point, you accuse me of not dividing by the number of iterations, as if the value wouldn't be a number of sub-second units. Then in your second point you do think it's a number of sub-second units. You can't have it both ways.

[reply]
[d/l]
[select]
        The reason the code assigns to a variable outside the scope of block is to avoid a smart optimizer optimizing the '&' or '% 2' away, as the result isn't used at all. I didn't want to dive into the Perl source code and see whether '&' or '%' checked their context and decide to do nothing in void context. If one or both would do so, the benchmark would be flawed. It's the same for an assignment to a variable that's about to go out of scope; if the variable doesn't have magic attached, one might not do the assignment after all. I've been bitten by the optimizations Perl makes so many times when benchmarking, I'm very careful and try to give the leave as little room for short cuts as possible. Hence my preference for assigning to a variable whose scope goes beyond the block being tested.

        As for your version, Perl parses ++$counter1 & 1 and 1 as '???' if ++$counter1 & 1. I don't know how much that means for the benchmark.

[reply]
[d/l]
[select]

        I do remember seeing that a long time ago, I've not encountered it with recent versions?

        The thing causing this behaviour is still there, running empty loops and subtracting times.

        Here's an example: 

        #!/usr/bin/perl

use strict;
use warnings;

use Benchmark;

timethese -1, {
    trivial1  => sub {1},
    trivial2  => sub {2},
};

__END__

Benchmark: running trivial1, trivial2 for at least 1 CPU seconds...
  trivial1:  1 wallclock secs ( 1.13 usr +  0.00 sys =  1.13 CPU) @ 44
+79726.55/s (n=5062091)
  trivial2:  3 wallclock secs ( 1.97 usr + -0.01 sys =  1.96 CPU) @ 43
+171185.71/s (n=84615524)

        [download]
        BTW, it took about 5 _minutes_ to run this benchmark. And for some reason, it decided to run 'trivial2' 16 times as many times as 'trivial1'.
[reply]
[d/l]

In Section Meditations