tofu has asked for the wisdom of the Perl Monks concerning the following question:

Hi -- I am a new user of MP2 seeking help. I posted this question earlier to the mod-perl list but not get an answer, so perhaps folks here can offer advice...

Here's my situation: My Apache generates 4 sets of logs. Three of them are generated by Apache directly: access, error, and a custom log tracking access for certain files. The fourth is made w/in a MP2 handler; here's the code: 

sub mylog {
    my (@fields) = @_;
    my $entry = join ( "|", @fields );
    open my $fh, ">>" . &MYLOG or die "can't open " . &MYLOG . ": $!";
    flock $fh, LOCK_EX;
    print $fh $entry . "\n";
    close $fh;
}

[download]

Currently I'm rolling my logs "manually" -- a process renames the four sets of logs, waits, then restarts the server, waits, moves away the old logs, etc.

I am considering using piped logs with rotatelogs instead to avoid the restart. I understand how to instruct Apache to log via a pipe to rotatelogs for access and error and custom.

My question relate to the mylog() function:

Thanks very much for your advice and insights....

TOFU

Replies are listed 'Best First'.
Re: open FH, "|foo" under modperl2
by perrin (Chancellor) on Feb 06, 2004 at 14:24 UTC
    I don't know anything about piped logs either, but I don't see why mod_perl would change the situation any.

    When I want to log something myself, I usually open the file for append (>>) which allows you to not bother with locking.

[reply]
Re: open FH, "|foo" under modperl2
by Anonymous Monk on Feb 06, 2004 at 21:08 UTC

    You've got a nasty little race condition in your log handling. Someone can squeeze in and modify the log file between the time you've opened the file for append and the time you've thrown a lock on the file. The simple cure for this is to toss in a seek $fh, 0, 2; after the call to flock().

[reply]
[d/l]
      Someone can squeeze in and modify the log file between the time you've opened the file for append and the time you've thrown a lock on the file. The simple cure for this is to toss in a seek $fh, 0, 2; after the call to flock().
      I thought "open for append" implied that the system takes care of this, all by itself?

      The next test seems to confirm that (Redhat Linux): 

      #!/usr/local/bin/perl -lw
my $file = "test.txt";
unlink $file;
if(fork) {
    open STDOUT, ">>$file" or die "Can't write to file: $!";
    $| = 1;
    print "first";
    sleep 3;
    print "third";
} else {
    open STDOUT, ">>$file" or die "Can't write to file: $!";
    $| = 1;
    sleep 1;
    print "second";
}

      [download]
      Result in the file: 
      first
second
third
[reply]
[d/l]
[select]
Re: open FH, "|foo" under modperl2
by Anonymous Monk on Feb 06, 2004 at 11:53 UTC
    I know nothing about rotatelogs but I would assume that if it is used with the CustomLog directive that Apache executes it once at startup and then pipes all log output through this one process. If you simply substituted a pipe to it in your open call then you would be executing it every time mylog() gets called, this will lead to increased overhead, not to mention the possibility that rotatelogs won't work like you want if it is repeatedly run like this (it might, for example, start a new log each time it is run rather than appending to the previous one). These problems would be solved by opening it once at startup and then just printing to it in mylog().

    Yes, you can open the pipe like a file

    flock only works on files (as opposed to filehandles), but if multiple processes are using the same pipe then you will probably need some sort of synchronization such as locking a sentinal file. When I say "using the same pipe" I am referring to creating the pipe outside of mylog() as mentioned earlier, but even if you were creating it in mylog() you might have multiple copies of rotatelogs writing to the same file depending on how rotatelogs works so a lock on a sentinal file would be a good idea here too.

    Your last two questions were answered with the first.
[reply]

      perlfunc:flock says:

      flock FILEHANDLE,OPERATION
      Calls flock(2), or an emulation of it, on FILEHANDLE.
      Admittedly, I've never tried to flock anything other than a filehandle related to a physical file, however to be picky: flock _does_ work against filehandles.

      If the information in this post is inaccurate, or just plain wrong, don't just downvote - please post explaining what's wrong.
      That way everyone learns.
[reply]
[d/l]

Back to Seekers of Perl Wisdom