Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks, I am trying to schedule a small perl script to clean a log file (empty it), trying to find the best approach to do this ,I need to back up the old one with current date before overwriting it . Trying something like : 
{    
    open my $wipe_file, '>', $filename or die $!;
    close $wipe_file or die $!;
    #done
}

[download]
this not the best way plus not backing up old file , any advice on doing this in a clean way ? thanks in advance

Replies are listed 'Best First'.
Re: Emptying log file
by imp (Priest) on Jan 31, 2007 at 16:52 UTC
    If possible I would use an existing utility for this, likely logrotate. If not then I would look to existing modules as they have probably done a reasonable amount of testing. At a glance Logfile::Rotate seems reasonable, but I don't have any personal experience with it.
[reply]
      There are a number of existing tools that can be used for logfile rotation (such as logrotate) and cleanup of old files (like tmpwatch). Those may or may not suit the purposes of the OP. If the requirement is to name the old log files based upon the date of the rotation, logrotate can accomplish that.

      The problem with rolling your own log file rotation script, as with other system utilities is that you need to take other issues into account, such as making sure that removal of old files does not remove the wrong file (such as removing /etc/passwd when /var/log/messages is a symbolic link to it.) Does your rotation script need to rotate the logs when the disks are filling up? As imp said, the existing tools have been well-tested, whatever tool you write will also need to be tested carefully.

[reply]
Re: Emptying log file
by sgifford (Prior) on Jan 31, 2007 at 17:16 UTC
    The tricky part is that if a program is actively writing to the log file, it will keep writing to the same file even after it's renamed. If there are any long-running processes writing, you will need to signal them to close the old log and open a new one (many existing daemons will do this when sent a particular signal).

    Another possibility is to use multilog, which deals with a lot of this automatically.


    --
    sgifford's Web page
[reply]
Re: Emptying log file
by kyle (Abbot) on Jan 31, 2007 at 17:04 UTC

    You can use rename to move the file once you decide on a place for it and then use your code to recreate an empty one. If you're putting the backup on another filesystem, you could use move from File::Copy.

[reply]
[d/l]

      You have to be careful with this though. File::Copy's move() will attempt to rename the file and, failing that (for example if the file is being moved across filesystem boundaries) will copy the file over and unlink the original. As sauoq quite rightly points out, this procedure runs the risk of losing log messages which are written in between the copy and the unlink.

      If you are trying to move a file to a different filesystem you should first rename it to a temporary location on the same filesystem as the original, then make sure that no more messages can be written to the renamed file (for example, by HUPing the daemon that logs to the file), then move the temporary file to it's final destination. You can use File::Copy for this last step if you want.


      All dogma is stupid.
[reply]
[d/l]
Re: Emptying log file
by sauoq (Abbot) on Jan 31, 2007 at 17:35 UTC
    any advice on doing this in a clean way ?

    As someone else pointed out, you should probably rename the file and create a new one. You may need to restart the process writing to the file in case it still has an open descriptor to the previous file. In fact, that's one big reason why it is better to rename rather than copy the file... you won't lose any log messages that way.

    -sauoq
    "My two cents aren't worth a dime.";
[reply]
Re: Emptying log file
by sgt (Deacon) on Jan 31, 2007 at 23:00 UTC

    sadly this is not an easy problem as you need cooperation from the tool that does the logging

    I suppose Un*x in the following. The main ideas probably apply to other OSes too.

    If the tool opens, writes log info, closes the logfile each time, then a move-truncate scheme is posible

     % mv $log $log.1 && > $log # mv-truncate

    If the tool opens just once but does "straight" appends and fluhes, a copy-truncate scheme is usually ok. There is a "logging window that you loose"

     % cp $log $log.1 && > $log # copy-truncate

    the rest which is tough :(

  • A bad case but common enough: daemon opens once, seeks to end of last write, writes log info. If you truncate you get holes that count as NUL (Ascii 0) bytes if you cp for example. This often happens when the tool has different logging modes, and one of them is "circular logging". By the way renaming the file is of no use as the running process holds a descriptor to it. Even deleting does not work you'll get an hidden file (that is the old unix trick for having hidden temporaries, you open and unlink just after)
  • In some cases you can use the fifo trick. Stop the daemon, substitute the logfile for a fifo (some systems might even have a special "/dev/log" driver) and some reader daemon, that dumps a file every x bytes say (perl is perfect here but youŽll need a reader manager is you use the scheme with more than a few processes). Restart the daemon, if it starts logging you are lucky (some processes do a -f test, start complaining and bail out).

    update: corrected typos.

    hth --stephan
[reply]
[d/l]
[select]
Re: Emptying log file
by glasswalk3r (Friar) on Feb 01, 2007 at 12:52 UTC

    I think we need to know some more details to help, like in which OS are you working and how the program that writes in the log file (for example, in append more, or if lock the files to keep writing at it).

    There are a lot of possibilities if you don't tell us the limitations of your system.

    Alceu Rodrigues de Freitas Junior
    ---------------------------------
    "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill
[reply]

Back to Seekers of Perl Wisdom