in reply to Is this the most efficient way to monitor log files..

When people talk about monitoring log files, the last thing you want to be doing is slurping the damned things into memory. Read them in line by line.

Taking graff's code, which seems like a good basis to work from, I would recast it as:

use strict; my %jobs; # start of main: open( IN, ".dractrack" ) or die "Can't open .dractrack for input: $!\n +"; my $ndone = 0; while( <IN> ) { chomp; my ($type,$primary,$rundir,$logfile,$host) = split; if ( not exists( $jobs{$rundir} )) { open( STAGE, "$rundir/jxrun.stg" ); my $lastrec; while( <STAGE> ) { $lastrec = $_; } close STAGE; $jobs{$rundir}{term} = (split(/\b/, $lastrec))[8]; } my $atstage = CheckStage( $rundir, $logfile ); if ( $atstage == $jobs{$rundir}{term} ) { print "$type job $primary on $host -- DONE"; $jobs{$rundir}{done}++; $ndone++; } else { printf("Cell: %s Verifying: %s at stage %3d of %3d -- %2.2f", $primary, $type, $atstage, $jobs{$rundir}{term}, 100 * $atstage / $jobs{$rundir}{term} ); } } close IN; # write a "current" version of ".dractrack", if necessary. # WATCH OUT! You REALLY need a semaphore or some other file # locking mechanism here (check Sean Burke's article about # semaphores in the most recent Perl Journal: # http://www.sysadminmag.com/tpj/) if ( $ndone ) { open( OUT, ">.dracnew" ) or die "Can't open .dracnew for output: $!\ +n"; open( IN, ".dractrack" ) or die "Can't open .dractrack for input: $! +\n"; while( <IN> ) { my $dir = (split)[2]; print OUT unless $jobs{$dir}{done}; } close DRAC and rename ".dracnew", ".dractrack" or die "Cannot rename .dracnew to .dractrack: $!\n"; } # end of main sub CheckStage { my ($path, $log) = @_; open( LOG, "$path/$log" ) or die "Cannot open $path/$log for input: $!\n"; my $at_stage_rec = undef; while( <LOG> ) { $at_stage_rec = $_ if /AT STAGE:/; } close LOG; $at_stage_rec ? (split / /, $at_stage_rec)[3] : undef; }

There are three places where files are being slurped: the main file, the stage files (and only the last line is needed!), and the log files, for which only the last line containing "AT STAGE" is needed. For all of these, a straight sequential read through the file will be able to pick up all that you need. This will be much cheaper than building up arrays left, right and center, only to throw them away after having picked out a single item.

Also, when checking whether open et al. fail, it is a good idea to also state what the error was, and any other additional information (file was trying to be opened for input or output), etc.


print@_{sort keys %_},$/if%_=split//,'= & *a?b:e\f/h^h!j+n,o@o;r$s-t%t#u'