I need to regulary monitor several hundred log files spread over approximately 100 servers (by regulary monitoring the logs I mean roughly every 120 seconds - I have a cron entry that runs my log check script every 2 minutes). Currently, I'm using File::Remote to open the files remotely. I use tell() to report the last line of the current open iteration of the file which I then write to disk (call it $position). I then use seek() to use this $position variable to jump to the last position checked in the file and continue processing (ie. checking for errors). My problem is two-fold: efficiency-wise I think I'm way off as the processing of each iteration of files takes longer than the 120 seconds that my cronjob is set up for. I could up the delay between script runs, but it's pretty important that errors are checked for at least every 2 minutes so I'd like to not go that route. Secondly, when traversing each file opened by File::Remote, the processing of the loop will hang without warning, often times on different files. My tests have all run after production hours so I've done my best to rule out production processes interfering with my logfile reading. Here's my code. Thanks for any help! --------------------------------------------------

#! /usr/bin/perl -w

use File::Remote;

# open error file for writing all of the log file(s)
# subsequent errors to
open(ERROR, '<error.txt') or die $!;

# open config file containing log file names and server
# locations
open(SERVER_LOGS, "<server_logs.txt") or die $!;

while (<SERVER_LOGS>) {
     chomp $_;
     # get server and logfile name for opening from file
     my ($server, $logfile) = split(/|/, $_);
     
     # open file ontaining the last file position read by
     # the script (unique file for each logfile)
     open(POSITION, "<${server}.${logfile}.line_position")
       or die $!; 
 
      my $position;
      while (<POSITION>) {
        $position = $_;
        chomp $position;
      }
      close(POSITION);

     # create new File::Remote object, backup remote 
     # logfile and open a file handle to it
     my $remote_fh = uc "${server}${logfile}";
     my $remote = File::Remote->new();
     $remote->backup($logfile, "sav");
     $remote->open($remote_fh, "<${server}:/log/ 
      ${logfile}") or die $!;

     # seek to last position in file that was read
     seek($remote_fh, $position, 0);

     # traverse particular logfile, write errors to error
     # file
     while (<$remote_fh>) {
        chomp $_;
        ### SCRIPT STALLS UNPREDICTABLY HERE ###
        if ($_ =~ /^ERROR/) {
            print ERROR "$_\n";
        }
     }

     # get current file position and write it back to
     # position file
     my $line_position = tell($remote_fh);
     open(POSITION, ">${server}.${logfile}.line_position")
       or die $!;
     print POSITION "$line_position";
     close(POSITION);
     $remote->close($remote_fh);
}
close ERROR;
[download]

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.