in reply to Benchmarking "Are all these characters in this sentence?"

I tried a little modified version of the RMGir_index one and the results are ... random. Especially since you do a lot more work in the benchmark than just the test you want to compare, the results depend on too many things and the actual different code seems to play only small part of it. I think you should at least change the benchmark_routine() to something like 

sub benchmark_routine
{
    my ($testFn, $testName)=@_;

    foreach(@testCases) {
        my ($sentence, $wantedLetters, $expectedResult) = @$_;
        $testFn->($sentence, $wantedLetters) for (1..20);
        die "$testName test failed ($sentence, $wantedLetters)"
          unless (($testFn->($sentence, $wantedLetters))==$expectedRes
+ult);
    }
}

[download]
to give a little more weight to the tested subroutine. Not that it helped too much in this case.

sub RMGir_indexNot {
    my ($sentence, $wantedLetters)=@_;

    # don't need this variable (or any of them, in
    # fact -- they're just here for clarity.
    # we could work straight out of @_ if we wanted
    # this terser
    # Also, the $[ check is just pedantic - if someone
    # changes $[, shoot them.
    return !(grep index($sentence,$_)<$[,
                                  split //,$wantedLetters);
}

[download]

Jenda
Support Denmark!
Defend the free world!

Replies are listed 'Best First'.
Re^2: Benchmarking "Are all these characters in this sentence?"
by RMGir (Prior) on Aug 29, 2008 at 09:58 UTC
    Good point. My assumption when I wrote the code was that the testFn routines were going to be fairly expensive, so benchmark_routine wasn't unacceptable overhead.

    I should have written separate test and benchmark routines, rather than let the benchmark run and check my test cases.

    Re-running the tests with the 1..20 fix you suggested does alter the percentages between tests, but doesn't change the relative rankings much. 

    Short

tallulah_OriginalPost             107/s
Tanktalus_AllRegex_Study          124/s
Tanktalus_AllRegex                130/s
varian_hash                       152/s
RMGir_slice                       207/s
moritz_BuildRegex_WithStudy       243/s
moritz_BuildRegex                 261/s
Tanktalus_AllIndex                452/s
RMGir_index                       678/s


LongShort                   

varian_hash                      4.76/s
RMGir_slice                      5.77/s
tallulah_OriginalPost             166/s
Tanktalus_AllRegex_Study          169/s
Tanktalus_AllRegex                207/s
moritz_BuildRegex_WithStudy       241/s
moritz_BuildRegex                 395/s
Tanktalus_AllIndex                731/s
RMGir_index                      1000/s


ShortLong

tallulah_OriginalPost            3.15/s
Tanktalus_AllRegex_Study         4.27/s
Tanktalus_AllRegex               4.31/s
moritz_BuildRegex_WithStudy      5.22/s
moritz_BuildRegex                5.26/s
varian_hash                      7.84/s
RMGir_index                      13.9/s
RMGir_slice                      15.3/s
Tanktalus_AllIndex               15.8/s


LongLong

tallulah_OriginalPost            3.17/s
Tanktalus_AllRegex_Study         4.24/s
varian_hash                      4.35/s
Tanktalus_AllRegex               4.35/s
moritz_BuildRegex                5.17/s
moritz_BuildRegex_WithStudy      5.22/s
RMGir_slice                      6.60/s
RMGir_index                      12.5/s
Tanktalus_AllIndex               15.5/s

    Mike
[reply]

In Section Meditations