watch-script.pl | less +F buffering problem?

axelrose has asked for the wisdom of the Perl Monks concerning the following question:

Hello all,

I wrote a little script to watch growing logfiles. It works nicely, just the output cannot be browsed if piped to less. Example usage:

terminal1 $ touch /tmp/my.log
terminal2 $ perl script.pl -p . /tmp/my.log | less +F
terminal1 $ echo "lala" >> /tmp/my.log
terminal1 $ echo "lala" >> /tmp/my.log
terminal1 $ ...
[download]

There is no visible output in terminal2, only if you interrupt less with ctrl-C.

#!/usr/bin/perl -w
use strict;

$| = 1;

use Getopt::Std;

my %opts;
getopts( 'p:f:s:h', \%opts );
my $pattern = $opts{p} || "pdf";
my $sleep = $opts{'s'} || 5;
my $logfile = shift || "/tmp/a.log";

my $fh = select( STDOUT ); $|=1; select( $fh );

print "watching file '$logfile' for pattern '$pattern'\n";

open(LOGFILE, "<" . $logfile) or die "not readable: '$logfile'$!\n";

seek(LOGFILE, 0, 2);
while (1) {
    while (<LOGFILE>) {
        print if /$pattern/o;
    }
    sleep $sleep;
    seek(LOGFILE, 0, 1);
}
[download]

Thanks for your time! Axel.

Comment on watch-script.pl \| less +F buffering problem? Select or Download Code

Replies are listed 'Best First'.
Re: watch-script.pl \| less +F buffering problem? by liverpole (Monsignor) on Aug 18, 2006 at 14:01 UTC
Hi axelrose, The problem is that less is buffering its input, so it expects to see either a certain number of bytes, or the end-of-file, before it will do any work. A better solution (though more work for you) would be to take the reason you want to use less (whether it be paginating the output, or searching for patterns), and incorporate that directly into your script instead. s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/	[reply]
Re^2: watch-script.pl \| less +F buffering problem? by axelrose (Scribe) on Aug 18, 2006 at 14:13 UTC
Hi liverpole, thanks for your reply. I thought about the input buffers of less as well but was surprised to see that `perl -e 'print "a line\n" x 10' \| less +F` works. Another workaround is to send the output of my watch script to another file and oberserving this with "less +F". My intention is to browse back the output over several pages and do occasional searches. I couldn't normally implement this myself. Axel.	[reply] [d/l]
Re^3: watch-script.pl \| less +F buffering problem? by liverpole (Monsignor) on Aug 18, 2006 at 14:30 UTC
That's because, in the case of: `perl -e 'print "a line\n" x 10' \| less +F` [download] the end-of-file is encountered as soon as the program finishes, so the pipe gets the end-of-file, and less knows it has received all the input it's going to get. Here's a good way to test it -- try executing the following command: `perl -e 'while (1) {print "a line\n" x 1000; sleep 1}' \| less` [download] Now the input buffer to less will fill up almost immediately, so you'll see the output right away. However, if you execute the command "G", it will never find the end-of-file, because the end-of-file hasn't been written yet! Similarly if you execute the same command with `less +G` at the end instead of `less`, you'll never see output for the same reason. s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/	[reply] [d/l] [select]
Re: watch-script.pl \| less +F buffering problem? by zentara (Cardinal) on Aug 18, 2006 at 13:44 UTC
Hi, I ran your script and it works fine with `echo "pdf" >> /tmp/my.log` [download] pdf appears in the watchers output, within a few seconds. `echo "lala" >> /tmp/my.log` [download] does nothing I'm not really a human, but I play one on earth. Cogito ergo sum a bum	[reply] [d/l] [select]
Re^2: watch-script.pl \| less +F buffering problem? by axelrose (Scribe) on Aug 18, 2006 at 13:54 UTC
That's presumably because you left out the command line parameter '-p .'. Then it checks per default for the string "pdf". If I just run the script without piping it to less all is fine here as well. It's just that somehow "script.pl -p . /tmp/my.log \| less +F" remains empty and it's not general problem because this: `perl -e 'print "a line\n" x 10' \| less +F` works. Axel	[reply] [d/l]