in reply to Tail'ing a log that frequently rolls over

The native port, tail.exe that comes with UnxUtils supports the -F switch (follow the name, not the descriptor) and that seems to work for this. I use it this way:

my $pid = open LOG, "tail -F theLog |" or die $!; while( <LOG> ) { ### }

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^2: Tail'ing a log that frequently rolls over
by frd1963 (Initiate) on Jan 09, 2009 at 15:56 UTC
    Great idea!
    I am a unix guy in general and knew that the GNU tail had the functionality I was looking for, but I didn't know there were individual, standalone native ports.
    I am Downloading that now and will let you know how it works out.

    Thanks!
    -Frd

Re^2: Tail'ing a log that frequently rolls over
by frd1963 (Initiate) on Jan 12, 2009 at 20:09 UTC
    Well it seems there are bugs with the UnxUtils version of tail as well. When using it in a perl script with any form of the "follow" options (e.g., "tail -f", "tail --follow", etc.) it never gives anything and the tail.exe process needs to be maually killed. "tail -f" works OK on the commandline except when its output is piped to another process (even "cat".) When run without the '-f' however, it works as one would expect.
    Still looking for a better solution and will update this thread once I find one.
    $fh->open("tail.exe --follow=name tls.log|") || die $!; print("opened: '$fh'\n"); while(my $c = $fh->getc) { # hangs here. tried getline also print("$c"); } print("closing: '$fh'\n"); close($fh);
      any form of the "follow" options (e.g., "tail -f", "tail --follow", etc.) it never gives anything

      Hm. It's possible you have a newer version that is fundementally broken and nobody has noticed, but I seriously doubt it. It is more likely to be how you are using it. The following two scripts demonstrate that it works:

      A simulated high speed, rotating logger:

      #! perl -slw use strict; my $logname = 'theLog'; while( 1 ) { open LOG, '>', $logname or die $!; for( 1 .. 1000 ) { print LOG "Line $_ of the log with some superfluous extra text + as filler"; } close LOG; my $n = 0; $n++ while -e "$logname.$n"; rename $logname, "$logname.$n"; }

      A tail -F script:

      #! perl -sw use strict; my $pid = open LOG, "tail -F theLog |" or die $!; $SIG{ INT } = sub{ close LOG; kill 3, $pid; exit; }; $SIG{ BREAK } = $SIG{ INT }; while( <LOG> ) { print; }
      the tail.exe process needs to be maually killed.

      Methinks you have unreal expectations. Any version of tail on any platform when used with -F (--follow=name) will have to be explicitly terminated, whether manually or programmically, by some external influence. It could not be any other way.

      When following a rotating named file, there will always be a short but real period between the renaming of the old log and the creation of the new one. Sp tail has to be written to ignore any 'file not found' errors from open and retry. There is therefore, no reasonable condition for it to self terminate.

      So your script would have to decide the conditions under which tail should be terminated and call kill to do so using the $pid returned by the open. (See the signal handler above.)

      So then it falls to your script to decide when it should terminate. Whether because

      • the user interrupted it from the keyboard or via some other action;
      • or because (for example,) it is monitoring the program that is producing the log and detects that it has terminated.

      If the latter, it will be necessary for you to detect the demise of the log producer in a timely manner, before re-entering the read state.

      You appear to be under the impression that tail will be able to know when the program producing the logs isn't going to create another new one, and self-terminate, but that simply cannot be the case.

      Perhaps that is the source of your problems with File::Tail and the POE solutions. Ie. Your expectations are wrong.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        I verified that your scripts fails for me as well. The only change I made was to swap the "tail -F" with "tail -f"

        Can you tell me where you got your tail.exe? Hopefully it is standalone.

        Thanks again, for at least helping me prove my own sanity ;)

        Thanks for the advice. I will be trying the code you provided to see if I get different results. I do notice that you use the "-F" option whereas the version of the tail.exe that I got from the above named site only accepts that in lower case. So we are obviously not using the same version of the utility, if even the same utility at all. Can you let me know where you got your version?

        Also, I wasn't clear in my expectations of tail. I know it wouldn't have any way of knowing the file was no longer being written to, not would I want it to; but I did expect that when its parent process died (perl) it would die as well. Of course, I had to forcefully kill perl to get it to exit which may be the equivalent of a UNIX kill -9 which would explain why tail never exited. It is easy enough to keep track of the PID in perl and kill it when deemed necessary, so I can put the appropriate code in there, but perl hangs at the read and so can't get to the code where it will kill the PID, unless from another thread or by handling an asycronous signal.

        Thanks again for the ideas and I will let you know what I find,
        -Frd