blue_cowdawg has asked for the wisdom of the Perl Monks concerning the following question:

Hey monks,

I was just presented with a very interesting problem that I think Perl might be able to solve, but I'm not 100% sure how to do it.

We have several processes on our production web application servers that spew to stdout. Currently we have stdout being redirected by our startup scripts for these processes to log files that when something goes wrong with the applications that run on top of these processes the developers review for errors.

The problem with this situation is two-fold.

  1. There is no time within a 24 hour period that the procesees can be restarted. Since the applications that run on top of these processes are involved directly with patient care their production periods are literally 24 hours a day. For that reason logrotate and friends won't work.
  2. The amount of spew these processes issue is on the order of several gig a day. The method that the developers use to access these logs is through a SAMBA share which gets bodgey when the logs get too large.
What I'm hoping to implement is some sort of filter that intercepts the stdout, occasionally closing the file handle (without losing the input stream) for the log file and moving it out of the way.

What schemes have others used to be able to rotate logs? Am I making this too complicated and is it as simple as: 

#!/usr/bin/perl -w 
use strict;

# starting psuedo-code
while(my $line=<>){
   print LOG $line;
   if file is too big...
         close LOG 
         rename file
         open LOG 
   }
}

[download]
or should I be thinking of other stuff besides?

Peter L. Berghold -- Unix Professional
Peter at Berghold dot Net
   Dog trainer, dog agility exhibitor, brewer of fine Belgian style ales. Happiness is a warm, tired, contented dog curled up at your side and a good Belgian ale in your chalice.

Replies are listed 'Best First'.
•Re: Log rotating filter.
by merlyn (Sage) on Jan 30, 2004 at 17:51 UTC
    Sounds like the apache rotatelogs will work just fine. You have your $unstoppable_process pipe its stdout to rotatelogs, and rotatelogs will scribble in different files on different days. Neither process ever needs to be stopped.

    Or, you can code your own in POE, having an alarm go off hourly or daily to rotate the output files while listening for new lines on standard input.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

[reply]

      As a slight variation on the Apache rotatelogs program, DJB's multilog might be worth a look. It works in a similar fashion, except that it rotates based on the size of the file and only keeps a set number of logfiles. This way you can guarantee that your log partition will not fill up.

      If you don't like the fact that it drops log files, you can set it up to mail them to you (not a good idea if you are talking gigs of data :) or do something else with the file that is about to be rotated out...

      It also allows you to filter out log entries based on patterns, and lets you send the output to several different log files based on the patterns.

[reply]
Re: Log rotating filter.
by arden (Curate) on Jan 30, 2004 at 18:01 UTC
    To guarantee that you never miss anything, this is what I've done on a similar sort of issue: 

    # I'll use psuedo-code since you did :^)
while (my $line=<>) {
    print LOG $line;
    if ( ++$linenumber == 90% of max ) {
        LOG == LOG1 ? open LOG2 : open LOG1
    }
    if ( $linenumber == max ) {
        LOG == LOG1 ? LOG = LOG2 : LOG = LOG1
        LOG == LOG1 ? close LOG2 : close LOG1
        LOG == LOG1 ? rename(LOG2) : rename(LOG1)
        $linenumber = 0
    }
}

    [download]

    Depending on how fast your logs fill up, you could push it up to as much as 98% of max before you bother to open the new log.

[reply]
[d/l]
Re: Log rotating filter.
by sfink (Deacon) on Jan 30, 2004 at 17:59 UTC
    I have done something very similar to this before, except that I had some different requirements that made things a bit more complicated. We were most concerned with the logs filling up our disk, so instead of rotating the logs, we wanted a "rolling" log that would keep the last 100MB of logs or so. Except that we also wanted to keep some of the log messages (based on severity) permanently.

    Still, from that experience, I can tell you that your pseudocode should work just fine. You don't necessarily have to rename the old log, though -- you could just reopen LOG with a new filename. Number them sequentially, or use some part of the time and date, or whatever. So something like (untested) this ought to work, as long as you don't need to restart your app using the same set of logs: 

    my $log_number = 0;
my $size = $MAXSIZE+1;
while (<>) {
    if ($size > $MAXSIZE) {
        # This will automatically close the old log file
        open(LOG, ">log-" . ++$log_number)
          or die "open: $!";
        $size = 0;
    }
    print LOG $_;
    $size += length($_);
}

    [download]
    Although if you wanted to be able to seek directly to some offset within the overall log file, you might be better off using a global $size counter and naming the log files by the offset of their starting byte, or perhaps an initial line number (you'd just need to keep a $next_start = $size + $MAXSIZE around).
[reply]
[d/l]
Re: Log rotating filter.
by Anonymous Monk on Jan 31, 2004 at 00:21 UTC
    Get Log::Dispatch::FileRotate from CPAN and you're all set: 
    use Log::Dispatch::FileRotate;

my $file = Log::Dispatch::FileRotate->new(
    name      => 'my_rotator',
    min_level => 'info',
    filename  => 'somefile.log',
    mode      => 'append' ,
    size      => 1024*1024,
    max       => 6);

while(<>) {
    $file->log(level => 'info', message => $_);
}

    [download]
    This keeps writing to somefile.log until it's 1MB. Then it will roll it over to somefile.log.1 and continue writing to a new somefile.log -- all transaction-safe. The 'max' parameter specifies the maximum number of rolled-over log files that you want to keep around this way until it starts throwing them away to avoid filling up the disc. It can also perform time-based rollovers (e.g. at midnight).

    This should get you what you need -- but another thought: if you wanted to do it the Right Way (tm), (and you had the luxury of being able to modify your web application) you'd have your application not write to STDOUT, but use something like Log::Log4perl -- this way you could configure any type of log destination just by modifying a configuration file: rotating appenders, database appenders, email appenders, you name it.
[reply]
[d/l]
Re: Log rotating filter.
by waswas-fng (Curate) on Jan 30, 2004 at 22:37 UTC
    If you are in a unix envirnment, you can always: 
    cp log.2 log.3
cp log.1 log.2
cp log.0 log.1
cp log log.0
cat /dev/null > log

    [download]
    There is a slight race situation on between the cp log log.0 and the cat /dev/null where you can lose a few lines of log. This method does not require the app that is dumping to the log to close and reopen the filehandle.


    -Waswas
[reply]
[d/l]

Back to Seekers of Perl Wisdom