I would try to process the log files in a single pass. You don't give a sufficient sample of your log files to be certain, but perhaps something like the following would work.

use strict; use warnings; my %transaction; foreach my $file (<*.log>) { open(my $fh, '<', $file) or die "$file: $!"; while(<$fh>) { chomp; if(/(.*?) \(([^\)]+)\)(.*)/) { my ($timestamp, $id, $log) = ($1, $2, $3); push(@{$transaction{$id}}, [ $timestamp, $id, $log, $file, + $. ]); my $status = status($transaction{$id}); if($status eq 'complete') { submit($transaction{$id}); delete($transaction{$id}); } elsif($status eq 'corrupt') { report($status, $transaction{$id}); delete($transaction{$id}); } } else { warn "invalid input line: $_"; } } close($fh); } foreach my $id (keys %transaction) { report('incomplete', $transaction{$id}); } exit(0); sub status { my $transaction = shift; return('incomplete') if(@$transaction < 3); return('corrupt') if( @$transaction > 3 or $transaction->[0][2] !~ /^(Authentication Request|Authenticati +on Response|Accounting Request)/ or $transaction->[1][2] !~ /^Acct-Session-Id :/ or $transaction->[2][2] !~ /^User-Name :/ ); return('complete'); } sub submit { my $transaction = shift; print "submit to database\n"; foreach my $line (@$transaction) { my ($timestamp, $id, $log, $file, $line) = @$line; print "\t$file line $line: $timestamp ($id)$log\n"; } } sub report { my ($status, $transaction) = @_; print STDERR "ERROR: $status transaction\n"; foreach my $line (@$transaction) { my ($timestamp, $id, $log, $file, $line) = @$line; print STDERR "\t$file line $line: $timestamp ($id)$log\n"; } }

In reply to Re: Interlaced log parser by ig
in thread Interlaced log parser by tzen

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.