in reply to Re: Re: Obtaining Apache logfile stats?
in thread Obtaining Apache logfile stats?

The quick and dirty approach would be to just carve it up on white space like you are doing with awk anyway. The conversion is straight forward. Use split or perl's -a option (as in my example above.)

Regardless of how you parse the input, you'll probably find it worthwhile to compute the statistics for every file accessed on one pass through your log. That's a lot more efficient than reading your whole log once for each file you want stats on. That's easy enough; just use a hash to maintain data for each filename as you traverse the log.

-sauoq
"My two cents aren't worth a dime.";

Replies are listed 'Best First'.
Re: Re: Re: Re: Obtaining Apache logfile stats?
by mvam (Acolyte) on Mar 25, 2004 at 23:14 UTC
    split is a good idea.. would it be faster to use an array?
[reply]
      would it be faster to use an array?

      If you mean "faster to use an array instead of a hash for collecting the data", then no, it would not be faster. I would split each line into an array during processing though.

      The idea is to key the hash by the filenames. So, everytime you come across, for instance, "/some/dir/file.html", you increase a count and a sum. The code might look something like this (untested): 

      while (<LOG>) {
    my @part = (split ' ', $_)[5,8]; 
    $hash{$part[0]}->[0] ++;             # increase the count. 
    $hash{$part[0]}->[1] += $part[1];    # increase the sum.
}

      [download]
      Note that the values of the hash are arrayrefs in order to store both the count and the sum associated with each filename. After you've munged your logs into raw data, you'll traverse the hash you created and compute the stats you want. Something like (again, untested): 
       
for my $key (sort keys %hash) {
    my $avg = $hash{$key}->[1] / $hash{$key}->[0];    # sum/count.
    print "$key\t$avg\n";  
}

      [download]

      -sauoq
"My two cents aren't worth a dime.";
[reply]
[d/l]
[select]
        ah ok, i think i get it. thanks sauoq!
[reply]

In Section Seekers of Perl Wisdom