in reply to Apache::ParseLog

I went through this drill a while back. I found that the best solution was installing a custom apache log handler that sends logs to a database. It works great. Here is an example script to do it. It was easy enough to look at this script and then write my own. I don't fool with parsing that silly access log anymore. What a pain.

Dave

Replies are listed 'Best First'.
Re: Re: Apache::ParseLog
by DrZaius (Monk) on May 01, 2001 at 18:45 UTC
    And you use this in production? Ouch.

    I have some code that isn't all that complex. It takes an access log and parses it and drops it into a heavily normalized database. From there you can run all sorts of queries against it.

    The hardest part is parsing the access log string; I haven't seen any good perl code ever (even the stuff I wrote :)) that does a perfect job. You're always going to miss some records -- especially because people do things like 'get /"' for some weird reason and throw off the quoting context.

    One option I haven't played with yet is using a funky delimiter between every entry, like |&| or something that should never occur in nature. This breaks a bunch of propietary software we use, so it isn't really an option here.

    My experience with Apache::ParseLog is that it does far too much. You should be able to feed it a format string and a filehandle -- how often do you parse your logs while they are sitting on a web server?

    out.