Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I need to read in a log file and 1st search for a string and then count back 8 lines to get the date. I need to repeat these steps until the end of the log file. I tried to read into an array and then use unshift. /Kevester

Replies are listed 'Best First'.
Re: jump lines in log file
by Utilitarian (Vicar) on Jul 20, 2009 at 15:12 UTC
    Suggested starting point
    • Read the file into an array
    • if $log_file[7]=~ /matches this/
    • then $date = $log_file[0]
    • else splice(@files,0,8) (Updated on the wise advice of moritz below)
    Code that up, and see if it works, if not we can have a look, however without code or sample data we can only really have a punt @ this, the line to match may have unique characteristics, the other records may be different lengths, etc..
[reply]
[d/l]
[select]
      else shift 8 times

      Or use splice to do it one go.

[reply]
Re: jump lines in log file
by kennethk (Abbot) on Jul 20, 2009 at 14:57 UTC
    If you tried to do the above, what didn't work? What code have you produced so far? Can you guarantee that none of these zones overlap, so you are not throwing away useful information with your shifts? Please read How do I post a question effectively? and remember this is not a code writing service.
[reply]
Re: jump lines in log file
by spazm (Monk) on Jul 20, 2009 at 16:25 UTC
    Tie::File is likely going to do a better job of dealing with large logs.
    DESCRIPTION

    Tie::File represents a regular text file as a Perl array. Each element in the array corresponds to a record in the file. The first line of the file is element 0 of the array; the second line is element 1, and so on.

    The file is not loaded into memory, so this will work even for gigantic files.

    Changes to the array are reflected in the file immediately.

    Lazy people and beginners may now stop reading the manual. 

    use Tie::File;
my $filename = 'thelog.log'
tie my @array, 'Tie::File', $filename 
    or die "can't open file:$filename : $!";

for my $i ( 0 .. $#array )
{
    my $line = $array[$i];
    if ( $line =~ m/MYLOGREGEX/x )
    {
         my $date = $array[ $i - 8 ];
         ## do something with $line and $date
    }

}

    [download]
    ps. dictated, but not read.
[reply]
[d/l]
Re: jump lines in log file
by gulden (Monk) on Jul 20, 2009 at 15:55 UTC
    Another aproach: 
    #!/usr/bin/perl -w
#
# UNTESTED CODE
#

use strict;
use warnings;



open(FILE, "file.log") or die "$!";

my ($date,$found);
while(my $line = <FILE>){
  chomp;


  if (/^(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}).*$/){ # if match Date
    $date = $1;   # set DATE
  }elsif ( /1st search for a string/ ){ # if match what i want
    $found = $line;
  }


  if($found){
    print "$date:$found\n";
    $found = undef;
    $date = undef;
  }
}

close(FILE);

    [download]
    «A contentious debate is always associated with a lack of valid arguments.»
[reply]
[d/l]
Re: jump lines in log file
by jwkrahn (Abbot) on Jul 20, 2009 at 16:09 UTC

    Something like this (UNTESTED) should work: 

    my ( @buffer, @data );
while ( <LOG_FH> ) {
    if ( /(DATE_PATTERN)/ ) {
        push @buffer, [ $., $1 ];
        shift @buffer if @buffer > 8;
        }
    if ( /SEARCH_PATTERN/ && @buffer && $buffer[ 0 ][ 0 ] == $. - 8 ) 
+{
        push @data, [ $buffer[ 0 ][ 1 ], $_ ];
        }
    }

    [download]
[reply]
[d/l]
Re: jump lines in log file
by hangon (Deacon) on Jul 21, 2009 at 08:07 UTC

    Another way is to set up a simple queue: 

    my $string = ... 
my @queue;

push @queue, scalar <$FH> for (1..8);

while (my $line = <$FH>){
    push @queue, $line;
    my $target = shift @queue;
    getdate($target) if $line =~ /$string/;
}

sub getdate{ ...

    [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom