in reply to Obtaining Apache logfile stats?

i wasnt able to get apache::parselog to work

From this comment and your data sample, I have to guess that you aren't using a standard log format, right? Can you show us a sample of your log data and/or the CustomLog directive you use in your Apache configuration? Without that, we can't help you slice and dice it in Perl.

is there a simple way with perl to parse the file and have the ability to pass a file name as an argument?

Yes. Something like the following might work well enough for you depending, of course, on what you haven't told us yet... 

#!perl -lan
BEGIN { 
    $SUM = $N = 0; 
    $file = shift; 
} 
if ($F[1] eq $file) { 
    my ($secs) = ($F[2] =~ /^(\d+)/); 
    $SUM += $secs; $N++  
} 

END { 
    print "Average: " , $SUM/ $N;
}

[download]
Put that in a file and run it with two arguments, the full pathname of the file you want stats on and the pathname of the file your parsed log data (i.e. the sample you provided) is in. Something like: 
perl get_stats /manual/misc/perf-tuning.html log.data

[download]

-sauoq
"My two cents aren't worth a dime.";

Replies are listed 'Best First'.
Re: Re: Obtaining Apache logfile stats?
by mvam (Acolyte) on Mar 25, 2004 at 21:46 UTC
    a sample log line:

    x.x.x.x - 24/Mar/2004:12:26:52 -0800 "GET /manual/misc/perf-tuning.html HTTP/1.1" 200 0 48296 "http://localhost/manual/" "Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.6) Gecko/20040211 Firefox/0.8"

    and this is my logformat line:

    LogFormat "%v %{x-up-subno}i %t \"%r\" %>s %T %b \"%{Referer}i\" \"%{User-Agent}i\"" wap

[reply]

      The quick and dirty approach would be to just carve it up on white space like you are doing with awk anyway. The conversion is straight forward. Use split or perl's -a option (as in my example above.)

      Regardless of how you parse the input, you'll probably find it worthwhile to compute the statistics for every file accessed on one pass through your log. That's a lot more efficient than reading your whole log once for each file you want stats on. That's easy enough; just use a hash to maintain data for each filename as you traverse the log.

      -sauoq
"My two cents aren't worth a dime.";
[reply]
[d/l]
[select]
        split is a good idea.. would it be faster to use an array?
[reply]

      How are you calculating the "0_seconds" portion of your sample data?

      -sauoq
"My two cents aren't worth a dime.";
[reply]

        sauoq, I believe that's coming from the "%T" portion of the log format.


        _______________
        DamnDirtyApe 
        Those who know that they are profound strive for clarity. Those who
would like to seem profound to the crowd strive for obscurity.
            --Friedrich Nietzsche
[reply]
        0_seconds is a sed substitution .. its really just 0 or 1 or whatever was returned.so it would be 0_seconds, 1_second, etc.. i was hoping it would be easier to read in the output file, but it doesnt really matter if its there or not since the average is used.
[reply]

In Section Seekers of Perl Wisdom