shemp has asked for the wisdom of the Perl Monks concerning the following question:

I ran across a weird (to me) issue today that really slows down a script im running. (in fact it never finishes) The script is a cgi-script. Here is the part that is slowing down, i know this because i can eliminate the slow down, but im wondering why it slowed down in the first place. 
...
    foreach my $field (sort @$field_order) {
        print "\t<TR>\n";
        print "\t\t<TH ALIGN=RIGHT>$field :</TH>\n";
        for my $i (0 .. $num_recs) {
            print "\t\t<TD ALIGN=LEFT>$records->[$i]->{$field}</TD>\n"
+;

        }

        print "\t</TR>\n";
    }

[download]
$records is an array of hashrefs of strings, nothing special. The inne loop contains only the one print statement, and that is where the slowdown occurs. If i add the following line, like so: 
...
    # heres the new line of code
    $records->[$i]->{$field} .= "";

    print "\t\t<TD ALIGN=LEFT>$records->[$i]->{$field}</TD>\n";
...

[download]
then this script runs fast.
I have also tried checking the lengths of each value : 
   length($records->[$i]->{$field})

[download]
and they all are reasonable. I have also taken refs to various parts of this data structure, and looked at it with Data::Dumper, and nothing unusual appears.

So, i am completely perplexed! - but i have found a workaround.
Any insight would be great. If you need more info, let me know.

A co-worker is trying to get mod_perl to work, so that may be it??

Replies are listed 'Best First'.
Re: Program slows dramatically!
by Abigail-II (Bishop) on May 28, 2003 at 23:33 UTC
    I can't think of anything obvious where appending an empty string to a value causes a dramatic speed up. One non-obvious thing I can think of is that some of the $records->[$i]->{$field} records are undefined, and you have a $SIG{__WARN__} installed that is taking a long time (or STDERR is redirected to something that takes a long time). But that is rather far fetched, and not very likely.

    You might have encountered an obscure bug in perl. Would you be able to write a small, stand alone program that exhibits this behaviour? Trying to create such a program is a dual cutting edge. If the bug is in your program, trying to find the minimal program exhibiting the unwanted behaviour often makes you see what bug you made; but if the bug is in perl, the people on p5p know where to look.

    Abigail

[reply]
[d/l]
[select]
      It was indeed the problem with undefined values! I replaced the unusual fixer line with this code, and things worked fine. 
          $records->[$i]->{$field} = "" if !defined $records->[$i]->{$field}
+;

      [download]
      The undefined values were getting in there because i was pulling the records from MySQL, and there are null fields.

      Thanks much.
[reply]
[d/l]
Re: Program slows dramatically!
by nite_man (Deacon) on May 29, 2003 at 06:43 UTC
    For showing up bottleneck in your programm try to use a Benchmark::Timer: 
        use Benchmark::Timer;
    $t = Benchmark::Timer->new(skip => 1);
    
    foreach my $field (sort @$field_order) {
        $t->start('tag');
        print "\t<TR>\n";
        print "\t\t<TH ALIGN=RIGHT>$field :</TH>\n";
        for my $i (0 .. $num_recs) {
            print "\t\t<TD ALIGN=LEFT>$records->[$i]->{$field}</TD>\n"
+;

        }

        print "\t</TR>\n";
        $t->stop;
    }   

    $t->report;

    [download]
    and Devel::Profiler: 
    $ perl -MDevel::Profiler your_script.pl

    [download]
    tnan see results 
    $ dprofpp

    [download]
    If you run your script under mod_perl you can use Devel::Profiler::Apache: 
    # in httpd.conf
 PerlModule Devel::Profiler::Apache;

 # or in startup.pl
 use Devel::Profiler::Apache;

    [download]
    Because it's very difficult to find out problem just to see a code,
    but using of profiler and benchmark help you very well.
    Good luck! 
          
--------------------------------
SV* sv_bless(SV* sv, HV* stash);
[reply]
[d/l]
[select]
Re: Program slows dramatically!
by BrowserUk (Patriarch) on May 28, 2003 at 23:57 UTC

    Can you tell how you are measuring the "dramatic slowdown"?

    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


[reply]
      Dramatic Slowdown: with the added "fixer" line, the browser has the new page in about 2 seconds. Without that line, the script NEVER finishes, or at least times out the browser.
[reply]

        That's pretty good evidence:)

        I tried several different ways to re-create the problem and the code without the .= '' was always approx. 40% faster than that without.

        If you want to try and track the cause down, then as Abigail-II suggested above, take a copy of your unfixed script and slowly remove stuff around the salient bit until the problem 'goes away'. If, in the course of doing that, the cause hasn't become obvious, post the reduced version that shows the problem, and let the 100's of pairs of eyes here feast upon it for a while.

        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller


[reply]
[d/l]

Back to Seekers of Perl Wisdom