in reply to Re^2: patchable settings (big readmores)
in thread patchable settings

Your benchmarks show that adding the DB update work to the new method makes it run about 30% faster. This is a red flag and you should explain it before accepting the results.

I did scan the code quickly and didn't see an obvious reason, but the code is hard to read from here.

- tye        

  • Comment on Re^3: patchable settings (benchmark red flag)

Replies are listed 'Best First'.
Re^4: patchable settings (benchmark red flag)
by demerphq (Chancellor) on Oct 17, 2004 at 07:29 UTC

    Umm, sorry I should have been more clear. The actual timings were done on two different machines. One set of benchmarks on my box here, the other set via Dumper Prompt. When I get more time Ill run both via dumper prompt so the runs per sec numbers are comparable (i was more interested in the relative percentages, not the r/p/s)


    ---
    demerphq

      First they ignore you, then they laugh at you, then they fight you, then you win.
      -- Gandhi

      Flux8


Re^4: patchable settings (benchmark red flag)
by demerphq (Chancellor) on Oct 17, 2004 at 17:17 UTC

    I reran all the tests from the Dumper prompt with the updated code I just posted in reply to ysth elsewhere in this thread. 'new_u' and 'old_u' are with DB updates, 'new' and 'old' are without.

    new, new_u, old, old_u, each for at least 2 CPU seconds... new: 2 wallclock secs ( 2.04 usr + 0.00 sys = 2.04 CPU) @ 50 +5.62/s (n=1031) new_u: 8 wallclock secs ( 2.09 usr + 0.05 sys = 2.15 CPU) @ 23 +6.45/s (n=508) old: 3 wallclock secs ( 2.12 usr + 0.00 sys = 2.12 CPU) @ 13 +0.82/s (n=278) old_u: 7 wallclock secs ( 2.05 usr + 0.06 sys = 2.12 CPU) @ 10 +2.49/s (n=217) Rate old_u old new_u new old_u 102/s -- -22% -57% -80% old 131/s 28% -- -45% -74% new_u 236/s 131% 81% -- -53% new 506/s 393% 286% 114% -- new, new_u, old, old_u, each for at least 2 CPU seconds... new: 3 wallclock secs ( 2.19 usr + 0.00 sys = 2.19 CPU) @ 50 +0.11/s (n=1094) new_u: 5 wallclock secs ( 1.94 usr + 0.12 sys = 2.06 CPU) @ 24 +6.30/s (n=508) old: 2 wallclock secs ( 2.13 usr + 0.00 sys = 2.13 CPU) @ 13 +5.50/s (n=289) old_u: 4 wallclock secs ( 2.10 usr + 0.03 sys = 2.13 CPU) @ 10 +6.43/s (n=227) Rate old_u old new_u new old_u 106/s -- -21% -57% -79% old 136/s 27% -- -45% -73% new_u 246/s 131% 82% -- -51% new 500/s 370% 269% 103% -- new, new_u, old, old_u, each for at least 2 CPU seconds... new: 2 wallclock secs ( 2.09 usr + 0.00 sys = 2.09 CPU) @ 51 +0.56/s (n=1065) new_u: 5 wallclock secs ( 1.89 usr + 0.13 sys = 2.02 CPU) @ 26 +0.94/s (n=528) old: 2 wallclock secs ( 2.14 usr + 0.00 sys = 2.14 CPU) @ 13 +9.21/s (n=298) old_u: 3 wallclock secs ( 2.01 usr + 0.03 sys = 2.04 CPU) @ 11 +1.33/s (n=227) Rate old_u old new_u new old_u 111/s -- -20% -57% -78% old 139/s 25% -- -47% -73% new_u 261/s 134% 87% -- -49% new 511/s 359% 267% 96% --

    Note that new_u() is still considerably faster than old_u(), and that new() has an even greater advantage over old().


    ---
    demerphq

      First they ignore you, then they laugh at you, then they fight you, then you win.
      -- Gandhi

      Flux8


      Another point is that Benchmark.pm tries hard to ignore most of the DB/network time so you need to look at the elapsed time. For a second I was puzzled because the second number was obscured:

      new_u: 8 wallclock secs (n=508) old_u: 7 wallclock secs (n=217) new_u: 5 wallclock secs (n=508) old_u: 4 wallclock secs (n=227) new_u: 5 wallclock secs (n=528) old_u: 3 wallclock secs (n=227)

      But after dividing, the wall-clock rates are (rather inaccurately):

      new_u 63.5 old_u 31 new_u 101.6 old_u 56.75 new_u 105.6 old_u 75.67

      Good work.

      - tye        

        Here is a patch to add the new vars code into Everything.pm, afaik its against a current Everything.pm


        ---
        demerphq

          First they ignore you, then they laugh at you, then they fight you, then you win.
          -- Gandhi

          Flux8