frostpuppet has asked for the wisdom of the Perl Monks concerning the following question:

Im pretty new to Perl but have seen how well it preforms for doing log file analysis. i currently have a file that Im having some trouble getting all the information out of that i want. the file will contain some data like this. 
##Flag##
some T:89898
data T:3434
here T: 9390
##CloseFlag##
some data 
##Flag##
some T:898
data T:34
here T: 990
again T:342
##Close Flag##

[download]
Currently I am able to parse out the garbage data between flags in a given log file. The problem im having now is that i want to grab the integer in the first line after the flag and then the integer in the last line before the ##close Flag## and computer the difference. This is made harder by the fact that i do need to count the number of lines between the two flags. Any help would be appreciated.

Replies are listed 'Best First'.
Re: Looking Backward While counting
by morgon (Priest) on Oct 15, 2010 at 21:03 UTC
    Here one try I hope you can take it as a starting point.

    I make the follwing assumptions:

    - it is "##closeFlag## (without blank)

    - your input is stored in a file called "in.txt"

    - the input file is small enough to fit into memory

    use strict;

# first slurp the file
my $content = do {
                    local $/;
                    open my $fh, "<", "in.txt" or die $!;
                    <$fh>
                 };

# then split it into chunks
my @chunks = $content =~ /##Flag##(.*?)##CloseFlag##/sg;

# now analyze the chunks
for my $chunk (@chunks) {
   my ($first, $last) = $chunk =~ /T:(\d+).*?(\d+)\D*$/s;

   # count number of newlines
   my $newlines = $chunk =~ tr/\n/\n/;

   print "first-last: ", $first - $last, "# of lines: ", $newlines-1, 
+"\n";
}

    [download]
[reply]
[d/l]

      Just a very minor point regarding your code. You don't need to tr/x/x/ when just counting characters; tr/x// will do.

      knoppix@Microknoppix:~$ perl -E '
> $str = q{banana};   
> say $str;
> $cnt = $str =~ tr{a}{};
> say $cnt;
> say $str;'
banana
3
banana
knoppix@Microknoppix:~$

      [download]

      Cheers,

      JohnGG

[reply]
[d/l]
[select]
Re: Looking Backward While counting
by jethro (Monsignor) on Oct 15, 2010 at 21:07 UTC
    An often used method for cases like this is to redefine the input record separator $/ (default value is \n) to the separator of the data chunks, in your case "##Flag##\n". Then each line you read will be a string containing the lines after the Flag. Now split on newline and process the individual lines or the whole string with regexes. For example:

    local $/="##Flag##\n";
open my $fh, "<", $filename or die $!;
while (my $chunk= <$fh>) {
  my @lines= split /\n/,$chunk;
  my ($firstvalue)= $lines[0]=~m/:(d+)/;
  ...

    [download]
[reply]
[d/l]
Re: Looking Backward While counting
by BrowserUk (Patriarch) on Oct 15, 2010 at 21:50 UTC

    Like this? 

    #! perl -slw
use strict;

my( $first, $nFirst, $last, $nLast );
while( <DATA> ) {
    if( /##Flag##/ ) {
        ( $first ) = <DATA> =~ m[T:\s*(\d+)];
        $nFirst = $.;
    }
    elsif( /##CloseFlag##/ ) {
        printf "%d lines diff: %d\n", $nLast - $nFirst +1, $last - $fi
+rst;
    }
    ( $last ) = m[T:\s*(\d+)];
    $nLast = $.;
}

=output
c:\test>junk70
3 lines diff: -80508
4 lines diff: -556
=cut

__DATA__
##Flag##
some T:89898
data T:3434
here T: 9390
##CloseFlag##
some data
##Flag##
some T:898
data T:34
here T: 990
again T:342
##CloseFlag##

    [download]
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

Back to Seekers of Perl Wisdom