in reply to count sort & output II

You're running this as a CGI script, are you not? If so, then this line: 
    unless (++$counter % 10) { print "Press Enter"; <STDIN> }

[download]
..won't work. Whoever suggested using this must have assumed you were running this script from the command line. If you want to get back only 10 results at a time from your CGI script, you need to either: Your CGI script can't store your hash in between HTTP requests, since it is unloaded from memory after every request1. I'll assume you don't want to wait 2 minutes for each 10 records, so I'd recommend writing to a cache file somewhere, storing all of the referrer => num records in your preferred sorted order. On each request to your script, just get the appropriate 10 lines from the file, if it exists, or else regenerate it if it doesn't exist. There should also be a mechanism to force a regeneration if the contents of the cache get stale.

You may want to do some debugging to see where the major slowdown is.. If it's the while loop, there's probably not a lot you can do, but if it's in the sorting and copying of the hash keys (BTW, how big does this hash end up?), you may want to consider a non-hash-based solution. Your for-loop has to make a copy of all the hash keys in memory, which may take a long time, considering your HTTP-referer strings are all probably fairly long. Other monks might have some good ideas about improving this portion of the code, but I'm at a loss at the moment.

Good luck,

blokhead

1: Of course, this is not true if your script is running under mod_perl, but it doesn't look like it

Replies are listed 'Best First'.
Re: Re: count sort & output II
by mkent (Acolyte) on Dec 19, 2002 at 22:16 UTC
    Thanks. You're right, my Apache server isn't using mod_perl and actually I didn't know that might make a difference, so thanks for the explanation.

    For a cache, this is what I came up with. I'm sure there's a better way, so any suggestions welcome. It doesn't speed up the processing at all.

    # Sort and print
for (sort {$referers{$b} <=> $referers{$a}} keys %referers) {
    if ($counter <= 10) {
       open (FILE, ">storage1.txt") || die('Could not open $storage1.t
+xt: $1');
       print "$_ - $referers{$_}<p>";
       print FILE "$_ - $referers{$_}<p>";
       ++$counter;
    } elsif ($counter > 10 && $counter <= 20) {
       if ($counter == 11) {
           print "<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<
+a href=\"storage2.txt\"><font color=\"FF0000\">Next</font></a><br>";
           print FILE "<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n
+bsp;<a href=\"storage2.txt\"><font color=\"FF0000\">Next</font></a><b
+r>";
           open (FILE2, ">storage2.txt") || die('Could not open $stora
+ge2.txt: $1');
           print FILE2 "$_ - $referers{$_}<p>";
           ++$counter;
        }
    } elsif ($counter > 20 && $counter <= 30) {
       if ($counter == 21) {
           print "<p>a href=\"storage2.txt\"><font color=\"FF0000\">Pr
+evious</font></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a h
+ref=\"storage3.txt\"><font color=\"FF0000\">Next</font></a><br>";
           open (FILE3, ">storage3.txt") || die('Could not open $stora
+ge3.txt: $1');
           print FILE3 "$_ - $referers{$_}<p>";
           ++$counter;
        }
     } elsif ($counter > 30 && $counter <= 40) {
       if ($counter == 31) {
           print "<p>a href=\"storage3.txt\"><font color=\"FF0000\">Pr
+evious</font></a><br>";
           open (FILE4, ">storage4.txt") || die('Could not open $stora
+ge4.txt: $1');
           print FILE4 "$_ - $referers{$_}<p>";
        }
     }
}
close FILE;
close FILE2;
close FILE3;
close FILE4;

    [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom