in reply to Process Log Files - Join Log entry pairs

while (<$FILE>){
   if (/^\d/){ 
       # new entry, sum values with those of $hash{$id}}and increment 
+$hash{$id}->count
   }
   elsif(/^\s+ID/){
                  @fields=split /\s+/;
                  $id=$fields[3];
    }
...

[download]
This type of structure would allow you to build your structure and then you can sum your values over the files by summing for each hash an element of the array (keys %hash).

Edit: fixed elsif typo Transient pointed out below.

Replies are listed 'Best First'.
Re^2: Process Log Files - Join Log entry pairs
by Transient (Hermit) on Jun 17, 2009 at 14:24 UTC
    A few things with that... while (<$FILE>) {
    would assume you had a variable that is a glob of a filehandle. I'm not sure why you'd want to do that, just a caveat.

    elif should be elsif

    and your code also assumes both a single line read (in the /^\d/ case) and a multi-line read (in the /^\s+ID/ case), unless I'm reading that incorrectly.

    Here's what I came up with: 
    use Data::Dumper;
use strict;

my $hash = {};

my @files = ( "LOG1", "LOG2", "LOG3" );

foreach my $file ( @files ) {
  open FILE, "<", $file or die "Unable to open $file\n$!\n";
   
  my ( $date, $id, $timestamp ) = qw( Unknown "" "" );
  while (<FILE>) {
     chomp;
     if (/^\d/){ 
        $date = $_;
     } elsif (/^\s+ID\s*=\s*(.*)$/) {
        $id = $1;   
     } elsif ( /^\s+TIMESTAMP\s*=\s*(.*)$/ ) {
        $timestamp = $1;
     } elsif ( /^\s+COUNT_\d+\s*=\s*(.*)$/ ) {
        $hash->{$date}->{$id}->{$timestamp} += $1;
     }
  }
}

print Dumper($hash), "\n";

    [download]
    which would give you 
    $VAR1 = {
          '20090619' => {
                          '4' => {
                                   '1244127600' => '20'
                                 },
                          '1' => {
                                   '1244127600' => '20'
                                 },
                          '3' => {
                                   '1244127600' => '10'
                                 },
                          '0' => {
                                   '1244127600' => '20'
                                 },
                          '2' => {
                                   '1244127600' => '10'
                                 }
                        }
        };

    [download]
    Assumptions are that the log files are all in the order specified, the actual COUNT_X values are immaterial, etc.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom