picabotwo has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to read in a log file on the fly. I need to buffer about 50 lines and match data in the log and perform an operation based up that match. Here is a sample transaction from the logfile. 
==============================================
CALL_INIT       2008/09/08 15:21:56.958
TRAN_ID         deadbeeffe3
CGI             310-26-1-1
CHAN            N/A
CHAN_MODE       Speech Version 2 (EFR)
CHAN_TYPE       TCH/EFR
SUB_CHAN        0
TIMESLOT        3
TSC             4
HOPPING         Enabled
MAIO            0
HSN             0
MA              528,529,530,531,532,533,534,542,543,544,545
ENCR            N/A
SESSION_KEY     N/A
RESPONSE_TIME   20000 ms
==============================================
CALL_RESPONSE           2008/09/08 15:21:59.223
TRAN_ID                 deadbeeffe3
CGI                     310-26-1-1
PLR_RESPONSE_TIME       20000 ms
Px,Py = bfbc76e04a8768e2,3faca2020f5a215b
UNCERTAINTY MAJOR       0x2 (2 m)
UNCERTAINTY MINOR       0x2 (2 m)
CONFIDENCE              70 %
TYPE OF SHAPE           3
LOCATION METHOD         8
==============================================

[download]
I basically have to look at the call init, and see if the channel type is TCH, when I see that I look for the Call Response with the same transaction ID, then I output the px, py to another file, or a c program. If the call init channel type is something else, I do nothing. Any ideas? Thanks! Josh

Replies are listed 'Best First'.
Re: Simple log parser I hope!
by tilly (Archbishop) on Sep 17, 2008 at 16:11 UTC
    Here is a possible main loop: 
    my %needed_transaction;
while (my %record = get_record()) {
  if ($record{CALL_INIT}
    and $record{CHAN_TYPE} =~ /TCH/
  ) {
    $needed_transaction{ $record{TRAN_ID} } = 1;
  }
  elsif ($record{CALL_RESPONSE}
    and $needed_transaction{ $record{TRAN_ID} }
  ) {
    output_record(%record);
    delete $needed_transaction{ $record{TRAN_ID} };
  }
}

    [download]
    The get_record function should read one block (up to the equals signs) and return a list of key/value pairs. The output_record function should pass along your Px, Py pair. I'll leave those to you to code.

    Personally I would pass around anonymous hash references rather than hashes, but I don't know if you know about references. If you want to try that, a good starting point is references quick reference. But as I say, you don't need to go that route to get a working solution.

[reply]
[d/l]
Re: Simple log parser I hope!
by moritz (Cardinal) on Sep 17, 2008 at 16:01 UTC
    My idea is that you start to code, and see how far you get. perlsyn, perlretut and perlopentut are your friends.

    Maybe it also helps to set $/ to "==============================================\n".

    When you get stuck somewhere, come back and ask more specific questions.

[reply]
[d/l]
Re: Simple log parser I hope!
by Limbic~Region (Chancellor) on Sep 17, 2008 at 22:38 UTC
    picabotwo,
    The difficulty in your question is that your limited data sample requires us to assume a lot. The following works with the data provided. 
    #!/usr/bin/perl
use constant MAX_OPEN_CALLS => 100;
use strict;
use warnings;

my $log = $ARGV[0] or die "Usage: $0 <log_file>";
open(my $fh, '<', $log) or die "Unable to open '$log' for reading: $!"
+;

my %call;

{
    local $/ = '==============================================';
    while (<$fh>) {
        my ($type, $id, $info) = parse_rec($_);
        if ($type eq 'INIT') {
            next if $info !~ /TCH/;
            die "Multiple calls with id '$id' without response" if exi
+sts $call{$id};
            $call{$id} = undef;
        }
        elsif ($type eq 'RESPONSE') {
            next if ! exists $call{$id};
            print "Call with id '$id' has a Px,Py of '$info'\n";
            delete $call{$id};
        }        
        die "Too many unanswered calls" if keys %call > MAX_OPEN_CALLS
+;
    }
}

sub parse_rec {
    my ($rec)   = @_;
    my ($type)  = $rec =~ /CALL_(\w+)/;
    $type       = '' if ! defined $type;
    my ($id)    = $rec =~ /TRAN_ID\s+(\S+)/;
    my $info;
    if ($type eq 'INIT') {
        ($info) = $rec =~ /CHAN_TYPE\s+(\S+)/;
    }
    elsif ($type eq 'RESPONSE') {
        ($info) = $rec =~ /Px,Py\s*=\s*(\S+)/;
    }
    $info       = '' if ! defined $info;
    return ($type, $id, $info);
}

    [download]

    Cheers - L~R

[reply]
[d/l]
      Hash table was exactly what I was looking for. Thanks again! Josh
[reply]
      Thanks that code worked exactly as I needed it. I just did a few small small tweaks to get it do to what I needed! 
      #!/usr/bin/perl
use constant MAX_OPEN_CALLS => 100;
use strict;
use warnings;

#my $log = $ARGV[0] or die "Usage: $0 <log_file>";
#open(my $fh, '<', $log) or die "Unable to open '$log' for reading: $!
+";

my %call;

{
    local $/ = '==============================================';
    while (<>) {
        my ($type, $id, $info) = parse_rec($_);
        print "$_\n\n";
        if ($type eq 'INIT') {
            next if $info !~ /TCH/;
            $call{$id} = 1;
        }
        elsif ($type eq 'RESPONSE') {
            next if ! exists $call{$id};
            print "Call with id '$id' has a Px,Py of '$info'\n";
            print "===============================================\n";
            (my @args) = $info =~ /(.{8})(.{8}),(.{8})(.{8})/;
            #print "@args\n";
            my $cprogram = `./decrypt @args`;
            print $cprogram;
            print "===============================================\n";
            delete $call{$id};
        }        
        die "Too many unanswered calls" if keys %call > MAX_OPEN_CALLS
+;
    }
}

sub parse_rec {
    my ($rec)   = @_;
    my ($type)  = $rec =~ /CALL_(\w+)/;
    $type       = '' if ! defined $type;
    my ($id)    = $rec =~ /TRAN_ID\s+(\S+)/;
    my $info;
    if ($type eq 'INIT') {
        ($info) = $rec =~ /CHAN_TYPE\s+(\S+)/;
    }
    elsif ($type eq 'RESPONSE') {
        ($info) = $rec =~ /Px,Py\s*=\s*(\S+)/;
    }
    $info       = '' if ! defined $info;
    return ($type, $id, $info);
}

      [download]
[reply]
[d/l]
Re: Simple log parser I hope!
by toolic (Bishop) on Sep 17, 2008 at 17:10 UTC
    Quick and dirty solution using a state variable: 
    use strict;
use warnings;

my $flag = 0;
while (<DATA>) {
    if (/^CHAN_TYPE\s+TCH/) { $flag = 1 }
    if ($flag) {
        if (/Px,Py/) {
            print;
            $flag = 0;
        }
    }
}

__DATA__

    [download]
    Read more... (2 kB)

    prints: 

    Px,Py = bfbc76e04a8768e2,3faca2020f5a215b

    [download]
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom