Re^3: find combinations

I stand corrected. I changed my benchmark code to

timethese
(-10, 
    {
        'tye'      => sub 
        { 
            $iter  = find_lances_tye   ( \%mechs, 5, 240, 10 );
            while ( my $lance = $iter->()  ) 
            {
                push @tye, join (",", sort @{$lance->{names}}) . "(" .
+ $lance->{weight} . " tons)\n" ;
            }
            @tye = sort @tye;

        },
        'holli'    => sub 
        { 
            @holli = find_lances_holli (5, 240, 10, keys %mechs );
            @holli = map { join (",", sort @{$_->{names}}) . "(" . $_-
+>{weight} . " tons)\n" } @holli;
            @holli = sort @holli;
            
        },
        'demerphq' => sub 
        { 
            @demerphq = find_lances_demerphq ( \%mechs, 5, 240, 10 );
            @demerphq = map { join (",", sort @{$_->{names}}) . "(" . 
+$_->{weight} . " tons)\n" } @demerphq;
            @demerphq = sort @demerphq;
        },
        
    }
);
[download]

so it constructs the same data-structure in all cases and then I get

C:\>benchmark.pl
Benchmark: running demerphq, holli, tye for at least 10 CPU seconds...
  demerphq: 10 wallclock secs (10.41 usr +  0.02 sys = 10.42 CPU) @  1
+.44/s (n=15)
     holli: 12 wallclock secs (12.23 usr +  0.00 sys = 12.23 CPU) @  0
+.33/s (n=4)
       tye: 10 wallclock secs (10.33 usr +  0.02 sys = 10.34 CPU) @  1
+.84/s (n=19)
[download]

Pretty similar to your outcome. Thanks for the clarification.

holli, /regexed monk/

Comment on Re^3: find combinations Select or Download Code

Replies are listed 'Best First'.
Re^4: find combinations by demerphq (Chancellor) on Sep 07, 2005 at 13:56 UTC
Unfortunately your benchmark is still not what I would consider to be "fair". tyes solution _still_ returns an iterator which you then unpack in the benchmark sub. Wheras your solution and my solution have to return the list on the stack twice. This has an observable cost, especially for large lists like the ones we are dealing with. IMO the three variants of find_lances_XXX() should have identical API's, i.e. they should take the same arguments, and return the same type of list. Anythign else means that you are benchmarking different behaviour and unconsciously weighting some of the solutions different from the others. Another problem with your reformulation is that you have put the list normalization routine inside the benchmark. IMO this is undesirable as it adds the cost of normalizing the list to the runtime of the solutions, when the normalization code is not actually needed for the routine to meet the specification given. Also, sorting is very sensitive to intitial conditions, so this may also subtly favour one of the solutions. Also if you are going to benchmark the code, please use my updated code as the original posting contains some unnecessary stuff, and uses the stack for simplicity when for speed you would normally use pass by reference. --- $world=~s/war/peace/g	[reply]

Replies are listed 'Best First'.

Re^4: find combinations
by demerphq (Chancellor) on Sep 07, 2005 at 13:56 UTC

Unfortunately your benchmark is still not what I would consider to be "fair". tyes solution _still_ returns an iterator which you then unpack in the benchmark sub. Wheras your solution and my solution have to return the list on the stack twice. This has an observable cost, especially for large lists like the ones we are dealing with. IMO the three variants of find_lances_XXX() should have identical API's, i.e. they should take the same arguments, and return the same type of list. Anythign else means that you are benchmarking different behaviour and unconsciously weighting some of the solutions different from the others.

Another problem with your reformulation is that you have put the list normalization routine inside the benchmark. IMO this is undesirable as it adds the cost of normalizing the list to the runtime of the solutions, when the normalization code is not actually needed for the routine to meet the specification given. Also, sorting is very sensitive to intitial conditions, so this may also subtly favour one of the solutions.

Also if you are going to benchmark the code, please use my updated code as the original posting contains some unnecessary stuff, and uses the stack for simplicity when for speed you would normally use pass by reference.

---
$world=~s/war/peace/g

[reply]