in reply to Speeding up data lookups

Benchmark and Profile your code, or you're tilting at windmills.

I prefer Devel::SmallProf.

-QM
--
Quantum Mechanics: The dreams stuff is made of

Replies are listed 'Best First'.
Re^2: Speeding up data lookups
by suaveant (Parson) on Sep 19, 2005 at 19:18 UTC
    Of course! :)

    I have used Devel:DProf before... I will look at SmallProf...

                    - Ant
                    - Some of my best work - (1 2 3)

      I like SmallProf because it does line by line.

      I needed more out of the post-processing script, so I rolled my own. I'm still debating whether to post it or not.

      Update: It's posted here

      -QM
      --
      Quantum Mechanics: The dreams stuff is made of

Re^2: Speeding up data lookups
by demerphq (Chancellor) on Sep 20, 2005 at 16:13 UTC

    Well, frankly id say that this is the last advice to give. Much better is a period of reflection and analysis, followed by some research. Then you put together a system using sound algorithms. Then finally, when you need to cream some extra speed out of the system, then you do profiling.

    The point is that if you are using Bubble Sort to reorder a million record file in random order you can profile it all you want without ever making it go as fast as a mergesort would have.

    IOW: think first, then profile.

    ---
    $world=~s/war/peace/g

      IOW: think first, then profile.
      Yes, exactly. But that's assuming that we took time in the first place.

      I would go further and suggest a conscious design phase in the beginning. If it's not a one-shot, and speed is important, then start with requirements, design to them, possibly experimenting when choosing between implementations.

      But given an existing system that runs too slowly, profile it now. If it's a bubble sort on a million records, that will be highlighted in million candle spotlights.

      The reason I suggest Devel::SmallProf is because it is more granular than the other oft-mentioned profilers. If the program is one giant subroutine, Devel::DProf will indicate that most of the time is spent in that routine. Devel::SmallProf will indicate which line of code is the worst, the next worst, etc. (for certain values of "line of code").

      So I agree with you, "Think before Do". But sometimes "Do" has already happened, and it's generally more efficient to the let instrumentation tell you where to start looking. It may be wrong, but I'd bet it's right often enough to warrant first action.

      -QM
      --
      Quantum Mechanics: The dreams stuff is made of

        Absolutely... in a system of any size there tend to be many different parts, and it is good to know where you are getting hit the hardest.

        Of course.. when I am playing in an area I am not too sure of I like to run some mockups and profile them to actually see if things I think will be faster actually are ;)

                        - Ant
                        - Some of my best work - (1 2 3)