http://qs1969.pair.com?node_id=1040813

chrestomanci has asked for the wisdom of the Perl Monks concerning the following question:

I have a fairly complex standalone script that I would like to make faster. I have already profiled it using Devel::NYTProf, and using the profiling results I have found and fixed a number of hotspots in the code.

I am now at a stage where there are some tuning parameters to adjust to try to get best performance. For example, on my DBIx::Class queries, which related resultsets should I pre-fetch.

At the moment I am trying to find the best settings for such parameters by doing multiple test runs with the profiler enabled, but with different settings for the parameters, and then comparing the html reports visually

The process works to show me large differences, such as a function taking 10 seconds under one set of settings, but only 5 under another, but the whole thing feels rather crude. For example there is no easy way to tell how time saved in one function has been moved to more time spent in other functions.

I have also considered using the Benchmark module to compare related runs, but it does not look like a good solution as it only considers overall execution time, and does not delve any deeper in to where my script spends it's time.

Is there a better way? Is there a feature of NYTProf that will let me compare the numbers from two runs of the same script (or very similar scripts) while suppressing statistically insignificant differences?

Replies are listed 'Best First'.
Re: Differential profiling
by BrowserUk (Patriarch) on Jun 26, 2013 at 16:28 UTC
    At the moment I am trying to find the best settings for such parameters by doing multiple test runs with the profiler enabled, but with different settings for the parameters, and then comparing the html reports visually

    Be aware that the vary act of profiling can significantly change the relative performance of different parts of your code in addition to an across the board slowdown.

    Profilers have a habit of adding a fixed overhead to every instrumented point which means they have a disproportionate affect in tight, fast loops and small subroutines and methods.

    It's often good to sanity check what you're doing by logging a few hires timestamps at critical junctures: each iteration of an outer loop; or the start and end of each major phase of the program. It's easy to spend your time chasing the dragon of pretty graphs only to find you've expended undue energies on insignificant parts of your program because of the disproportionate affects of the profiling probes.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
Re: Differential profiling
by BrowserUk (Patriarch) on Jun 26, 2013 at 20:24 UTC

    BTW. Have you seen the various .csv files in the nytprof directory after a run?

    If you rename that directory between runs (to preserve the previous run stats, you can easily knock up a script that reads the relevant pair(s) of .csv files from the two runs and presents them interleaved with numerical differences so you can see what changed where between runs.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

Back to Seekers of Perl Wisdom