Re: Nested evals - are they evil? (sample benchmarks)

OK, I just made up a quick test case:

use strict;
use warnings;
use Benchmark 'cmpthese';

cmpthese(10000, {
    'eval'   => sub { eval_code() },
    'noeval' => sub { no_eval_code() },,
});

sub eval_code {
    my $x = 0;
    for (1..1000) {
        eval {
            $x+=1;
        }
    }
}

sub no_eval_code {
    my $x = 0;
    for (1..1000) {
        $x+=1;
    }
}
[download]

And, unless I've missed something blindingly obvious, in this trivial case there is quite an overheard with eval:

         Rate   eval noeval
eval   1495/s     --   -62%
noeval 3953/s   164%     --
[download]

But let's tweak that for a real life example. Here's some code where it "dies" one time in ten:

use strict;
use warnings;
use Benchmark 'cmpthese';

cmpthese(1000000, {
    'eval'   => sub { eval_code() },
    'noeval' => sub { no_eval_code() },,
});


sub eval_code {
    my $x = 0;
    for (reverse 0..9) {
        eval {
            my $x = 100/$_;
        };
        if ($@) {
            error_sub();
        }
    }
}

sub no_eval_code {
    for (reverse 0..9) {
        if ($_==0) {
            error_sub();
        }
        else {
            my $x = 100/$_;
        }
    }
}
[download]

and here's the benchmark results:

           Rate   eval noeval
eval    67797/s     --   -39%
noeval 111111/s    64%     --
[download]

Well. That's still significantly faster. So have I proved eval slows things down, or is this a case specific example that someone can find a counter-example for? I've been bitten by benchmarks on code snippets before, so I'm still wary of this result. Hmmmmm....

Comment on Re: Nested evals - are they evil? (sample benchmarks) Select or Download Code

Replies are listed 'Best First'.
Re^2: Nested evals - are they evil? (sample benchmarks) by GrandFather (Saint) on Jul 31, 2007 at 05:09 UTC
Exceptions should be "exceptional". That is, you shouldn't expect them to happen during normal functioning of the program. So lets see what the overhead of providing (unused) exception handling is: `use strict; use warnings; use Benchmark 'cmpthese'; cmpthese(-1, { 'eval' => \&eval_code, 'noeval' => \&no_eval_code, }); sub eval_code { my $x = 0; for (reverse 0..9) { eval { some_more_code ($_); }; if ($@) { error_sub(); } } } sub no_eval_code { for (reverse 0..9) { if ($_==0) { error_sub(); } else { some_more_code ($_); } } } sub some_more_code { } sub error_sub { }` [download] Prints: `Rate eval noeval eval 185563/s -- -7% noeval 198639/s 7% --` [download] In words: the processing cost of an unexercised eval is essentially nothing. DWIM is Perl's answer to Gödel	[reply] [d/l] [select]
Re^3: Nested evals - are they evil? (sample benchmarks) by cLive ;-) (Prior) on Jul 31, 2007 at 07:05 UTC
Hmmm, well, I stripped it down further: `use strict; use warnings; use Benchmark 'cmpthese'; cmpthese(-1, { 'eval' => \&eval_code, 'noeval' => \&no_eval_code, }); sub eval_code { eval { some_more_code (); }; if ($@) { error_sub(); } } sub no_eval_code { if (0) { error_sub(); } else { some_more_code (); } } sub some_more_code { } sub error_sub { }` [download] And got this (I think the effectively empty eval is meaningless - see here where I ran it 4 times): `Rate eval noeval eval 1298354/s -- -37% noeval 2066450/s 59% -- Rate eval noeval eval 1316991/s -- -38% noeval 2123851/s 61% -- Rate eval noeval eval 1217925/s -- -21% noeval 1534287/s 26% -- Rate eval noeval eval 1298354/s -- -41% noeval 2205537/s 70% --` [download] I have a feeling an empty eval gets optimized away at compile time (though I am most probably wrong :) How about if I add in an increment do ensure each eval has to be 'eval'd, ie `use strict; use warnings; use Benchmark 'cmpthese'; my $x; cmpthese(-1, { 'eval' => \&eval_code, 'noeval' => \&no_eval_code, }); sub eval_code { $x=1; eval { some_more_code (); }; if ($@) { error_sub(); } } sub no_eval_code { if ($x==0) { error_sub(); } else { $x=0; some_more_code (); } } sub some_more_code { $x++ } sub error_sub { }` [download] I get this: `Rate eval noeval eval 989400/s -- -23% noeval 1286220/s 30% -- -bash-3.00$ perl tmp.pl Rate eval noeval eval 1092266/s -- -21% noeval 1379705/s 26% -- Rate eval noeval eval 1069351/s -- -19% noeval 1327443/s 24% -- Rate eval noeval eval 1071850/s -- -25% noeval 1435550/s 34% -- Rate eval noeval eval 1071850/s -- -26% noeval 1456355/s 36% --` [download] Either way, I still haven't been able to create a counter example that shows eval to be faster. But anyway, considering the number of runs a second, I have a feeling my energies are probably better spent focusing on other parts of the code as far as optimizations are concerned :)	[reply] [d/l] [select]
Re^2: Nested evals - are they evil? (sample benchmarks) by perrin (Chancellor) on Jul 31, 2007 at 17:24 UTC
I would expect eval to be a little slower, but not that much. Maybe this is because of the extra scope you're adding here. Try changing the no_eval_code to this, to see if the extra scoping is free or not: `sub no_eval_code { for (reverse 0..9) { if ($_==0) { error_sub(); } else { { my $x = 100/$_; } } } }` [download]	[reply] [d/l]
Re^3: Nested evals - are they evil? (sample benchmarks) by cLive ;-) (Prior) on Jul 31, 2007 at 17:57 UTC
heh `Rate eval noeval eval 88768/s -- -39% noeval 146161/s 65% --` [download]	[reply] [d/l]


Clear questions and runnable code get the best and fastest answer
	PerlMonks