in reply to Obtaining Apache logfile stats?

ok here we go: the problem i need to solve is taking an apache log file thats rotated daily and getting the time taken to serve each page in the log. i was able to get this data in a basic format using 
awk '{print $3, %6, $9}' > /tmp/resultsfile

[download]
this does a nice job of outputting the relevant fields in the log file. the next step would to be process this output in such a way that i could type say 'mod_rewrite.html' and find out how many times it was served and what the average of those time is.

Replies are listed 'Best First'.
Re: Re: Obtaining Apache logfile stats?
by DamnDirtyApe (Curate) on Mar 25, 2004 at 20:35 UTC

    Alright, perhaps try something along these lines:

    #! /usr/bin/perl
use strict;
use warnings;

my $file = shift @ARGV;
my @times = map { /(\d+)_seconds/; $1 }
            grep { /$file/ } <DATA>;
my $totaltime; $totaltime += $_ for @times;
my $avgtime = $totaltime / @times;
print "Average time: $avgtime\n\n";

__DATA__
[24/Mar/2004:12:26:52 /manual/misc/perf-tuning.html 0_seconds
[24/Mar/2004:12:27:33 /manual/mod/mod_status.html 0_seconds
[24/Mar/2004:12:27:39 /manual/mod/module-dict.html 0_seconds
[24/Mar/2004:12:27:46 /manual/misc/rewriteguide.html 0_seconds
[24/Mar/2004:12:27:53 /manual/mod/mod_rewrite.html 5_seconds
[24/Mar/2004:12:27:53 /manual/images/mod_rewrite_fig1.gif 0_seconds
[24/Mar/2004:12:27:53 /manual/images/mod_rewrite_fig2.gif 0_seconds
[24/Mar/2004:12:28:05 /manual/new_features_1_3.html 0_seconds
[24/Mar/2004:12:29:53 /manual/mod/mod_rewrite.html 6_seconds
[24/Mar/2004:12:29:54 /manual/mod/mod_rewrite.html 7_seconds
[24/Mar/2004:12:29:55 /manual/mod/mod_rewrite.html 8_seconds
[24/Mar/2004:12:29:56 /manual/mod/mod_rewrite.html 9_seconds

    [download]

    I still think you should try the format manipulation in Perl, though; it's easy to do, and you'll only have one script to maintain.


    _______________
    DamnDirtyApe 
    Those who know that they are profound strive for clarity. Those who
would like to seem profound to the crowd strive for obscurity.
            --Friedrich Nietzsche
[reply]
[d/l]
      this did produce the average time, but ended up with

      Use of uninitialized value in regexp compilation at ./avgtime.pl line 6, <DATA> line 12.

      this repeated for each line in DATA. i'm a perl moron as you can see, but i'm trying. the down side to these log files is that they can reach 2GB in a matter of hours so creating the temp result file can get somewhat expensive. i'm thinking about grepping out anything with a zero value since really we only want to see when the server has a load
[reply]

        Did you pass in the filename to average on the command line?

        perl ./avgtime.pl mod_rewrite.html

        [download]

        Sorry, forgot to mention that - figured you'd pick it up from the code.


        _______________
        DamnDirtyApe 
        Those who know that they are profound strive for clarity. Those who
would like to seem profound to the crowd strive for obscurity.
            --Friedrich Nietzsche
[reply]
[d/l]

In Section Seekers of Perl Wisdom