neurotoxx has asked for the wisdom of the Perl Monks concerning the following question:

How can I search for a pattern in a log file, and then print the three lines before the line that has the matched pattern? For example:

Log file:

log info line yada yada 1 log info line yada yada 2 log info line yada yada 3 log info line (pattern) 4 log info line yada yada 5 ...

Results file:

log info line yada yada 1 log info line yada yada 2 log info line yada yada 3 (pattern) (Put line number here, or some type of result mark) ...

Here's the code I'm working with so far.

open(INFILE, "< file.log") or die "Cant open file : $!"; open(OUT, "> results.txt") or die "Cant open new file : $!"; $pattern = "Error code 2"; while (<INFILE>) { $a[$i++%3] = $_; print OUT $a[$i%3] if /$pattern/; } close INFILE; close OUT;

I saw a thread regarding the use of the "%3", but not sure how to use it correctly.

Please let me know how I could produce a result file that pulls the three lines prior to the match of the search string line.

Thank you in advance for any help that can enlighten me

-neurotoxx

Replies are listed 'Best First'.
Re: How can I print three lines before pattern match?
by ikegami (Patriarch) on Aug 14, 2009 at 02:55 UTC
    If you want to avoid loading the entire file into memory,
    my @history; while (<>) { print @history if /pattern/; push @history, $_; shift @history if @history > 3; }

    No direction was given for handling overlapping matches.

      Or if you don't want to test the length of your array forevermore:

      my @history = ("\n") x 3; while (<>) { print @history if /pattern/; push @history, $_; shift @history; }

        This works very well, thank all of you for your help.

        Where can I place a print statement that seperates the output results for each match?

        #!/Perl/bin/perl -w open(INFILE, "< file.log") or die "Cant open file : $!"; open(OUT, "> results.txt") or die "Cant open new file : $!"; use strict; my $pattern = "Error while detecting new item"; my @history = ("\n") x 3; while (<INFILE>) { print OUT @history if /$pattern/; push @history, $_; shift @history; } print $.; close INFILE; close OUT; exit;

        Example:

        results.txt matched line 1 matched line 2 matched line 3 results of first match matched line 1 matched line 2 matched line 3 results of second match and so on...

        Thank you again so much, Neurotoxx

        ++, but that should be my @history = ('') x 3;
Re: How can I print three lines before pattern match?
by bichonfrise74 (Vicar) on Aug 14, 2009 at 02:04 UTC
    Try this...
    #!/usr/bin/perl use strict; my @file = <DATA>; for my $i (0 .. $#file) { if ( $file[$i] =~ /pattern/ ) { print "$file[$_]" for ($i-3 .. $i-1); } } __DATA__ log info line yada yada 0 log info line yada yada 1 log info line yada yada 2 log info line yada yada 3 log info line (pattern) 4 log info line yada yada 5
      That doesn't work if the pattern is found in the first three lines. (And what's with the quotes around a variable?) Change
      print "$file[$_]" for ($i-3 .. $i-1);
      to
      print $file[$_] for grep $_ >= 0, $i-3 .. $i-1;
        Thanks ikegami. Those are good points.

      Not sure I understand this:

      for my $i (0 .. $#file)

      Is the #file a reference to my data file?

      So then, for my $i (0 .. $data.log)

      Thank you.

      Sorry for my ignorance.

        Is the #file a reference to my data file?

        No, not directly. Instead, the idea presented was to capture the whole file in @file:

        my @file = <DATA>;

        In your case, it would be <INFILE> instead of <DATA>. (<DATA> is a special file handle, starting after the special tag __DATA__ in the script, but otherwise pretty much a normal-behaving file handle.)

        The $# in front of an array identifier means "the last index of" that array.

        Perhaps better is to keep the last three lines in a fifo:

        open(INFILE, "< file.log") or die "Cant open file : $!"; open(OUT, "> results.txt") or die "Cant open new file : $!"; $pattern = "Error code 2"; my @file; while (<INFILE>) { print OUT @file if (/$pattern/ and @file); push @file, $_; # push on the current line shift @file while (@file > 3); # shorten to 3 elements } close INFILE; close OUT;

        Don't forget warnings and strict.

        -QM
        --
        Quantum Mechanics: The dreams stuff is made of

        $#file is the index of the last element in the array @file.
Re: How can I print three lines before pattern match?
by busunsl (Vicar) on Aug 14, 2009 at 07:25 UTC
    Do you have to use Perl?
    Of course it has to be Perl! ;-)

    Otherwise grep will do it:

    grep -B3 pattern logfile

      Unfortunately, I need to have it run as a Windows task. I also need it to traverse the log directory and process each file with the pattern match.

      Thanks,

      Neurotoxx