This is an update to see if anyone can suggest ways to speed up this script, which tends to be rather slow:

use strict; use warnings; use Date::Manip; use CGI qw/:standard/; # Make sure security is not compromised by calling unpathed programs. $ENV{PATH} = "/bin:/usr/bin:/usr/local/bin:"; $ENV{IFS}=""; # Use CGI to print the header print header; # Make variables local only my %referers = (); my $row = 0; my $counter = 0; # Retrieve and security-check parameters my $site = param('site'); my $hour = param('hour'); my $minute = param('minute'); if ($hour !~ /^\d\d?$/) { die('Invalid hour'); } if ($minute !~ /^\d\d?$/) { die('Invalid minute'); } # Get date object for the checkpoint my $check_date = ParseDate("${hour}hours ${minute}minutes ago"); # Select the server log - current 12/19/02 my $data = ''; if ($site eq 'star') {$data = 'indystar/access_log'} elsif ($site eq 'topics') {$data = 'topics/access_log'} else {$data = 'noblesville/access_log'} # Create headline for web page print "<h1>Referrers in the past $hour hours and $minute minutes</h1>" +; # File handling, one line at a time; if can't open, say why open(FH,"$data") || die('Could not open $data: $1'); while (defined(my $line = <FH>) ) { next if ($line !~ /^\S+ \S \S \[(\S+) \S+\] "[^"]+" \d+ \d+ "([^"] ++)"/); my $line_date = ParseDate($1); # Check to see if the line date is in the range we're after next unless Date_Cmp($line_date, $check_date)>0; # If the referer is new, set to 1 entry, otherwise increment if (not exists $referers{$2}) { $referers{$2}=1; } else { $referers{$2}++; } } close(FH); # Sort and print for (sort {$referers{$b} <=> $referers{$a}} keys %referers) { if ($counter <= 10) { open (FILE, ">storage1.txt") || die('Could not open $storage1.t +xt: $1'); print "$_ - $referers{$_}<p>"; print FILE "$_ - $referers{$_}<p>"; ++$counter; } elsif ($counter > 10 && $counter <= 20) { if ($counter == 11) { print "<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;< +a href=\"storage2.txt\"><font color=\"FF0000\">Next</font></a><br>"; print FILE "<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n +bsp;<a href=\"storage2.txt\"><font color=\"FF0000\">Next</font></a><b +r>"; open (FILE2, ">storage2.txt") || die('Could not open $stora +ge2.txt: $1'); print FILE2 "$_ - $referers{$_}<p>"; ++$counter; } } elsif ($counter > 20 && $counter <= 30) { if ($counter == 21) { print "<p>a href=\"storage2.txt\"><font color=\"FF0000\">Pr +evious</font></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a h +ref=\"storage3.txt\"><font color=\"FF0000\">Next</font></a><br>"; open (FILE3, ">storage3.txt") || die('Could not open $stora +ge3.txt: $1'); print FILE3 "$_ - $referers{$_}<p>"; ++$counter; } } elsif ($counter > 30 && $counter <= 40) { if ($counter == 31) { print "<p>a href=\"storage3.txt\"><font color=\"FF0000\">Pr +evious</font></a><br>"; open (FILE4, ">storage4.txt") || die('Could not open $stora +ge4.txt: $1'); print FILE4 "$_ - $referers{$_}<p>"; } } } close FILE; close FILE2; close FILE3; close FILE4;
Edit davorg - changed title

In reply to Log munging optimization tips? by mkent

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.