Greetings wise brothers.

In my day to day perl projects, I mostly use Log4perl for logging. It is flexable and it works quite nicely.

Recently I have joined a complex project involving a cluster of dozens of servers, each running several different daemons. The cluster processes arround 100_000 jobs per week, with different parts of each job running on different servers in the cluster. Log files are stored localy on each server. Each daemon logs to stdout which is redirected to a log file.

Needless to say it is difficult to investigate issues when stuff goes wrong as there are many log files on different servers to look at, and the only way to isolate log messages that relate to a particular job is to use grep. The volume of the log files is also a problem. Currently we log a lot of detail as it might be needed for later anasys, but this makes for huge log files and slow greping of them.

For that reason I am investigating putting the log messages into a database using Log::Log4perl::Appender::DBI or such like, as this would have the advantage of putting all the logs in one place, and make it easier to purge verbose messages on a schedule while keeping errors for much longer.

My problem is how to view and filter the logs in a user freindly way. While in theory I could write some SQL like: Select * from logs where jobID = 12345 in practice it is not a user friendly way of doing things.

I have seen that log4javascript has a web interface tool for log messages that lets you search and filter log messages. (See their demo). Does anyone know of a web interface that can be used to view and filter log messages from a database? My google searches are coming up empty.