in reply to Structuring maillog data - hopefully simple question on arrays/hashes

Thanks to the excellent input I received, I made something that appears to work - it may be ugly, but it's saved me a big job. The interesting part is that the client host (originating server) is only mentioned with reference to the first message ID, and the external delivery status only goes with the second message ID. The line that has both IDs comes halfway through the logging sequence (and other unrelated message transactions are interpolated in between). 

#!/usr/bin/perl
use warnings;
use strict;
my @logfiles = qw(maillog.4 maillog.5 maillog.6 [..]);
my $line;
my $file;
my $key;
my $value;

open (CLIENTOUT, ">>client.txt") or die "Cannot open output file: $!";

for $file ( @logfiles )  {
    open (FH, $file ) or  die "Cannot open file: $!";
    print "$file\n";
    my %msgids;
    while (my $line = <FH>) {
        chomp($line);
        if  ( $line =~ /client=(ourhost1|ourhost2)/ ) {
            no warnings 'uninitialized';   
#I don't care about warnings about something uninitialised in line 23
            my @id = ($line =~ /\_([0-9A-F]{11,12})/);
            $msgids{ $id[0] } = ();
        }
        if ( $line =~ /to=<.*\.gov>.*relay=localhost/ ) {
            if ( $line !~ /to=<.*(ourdomain\.gov)/i) {
                while ( ($key, $value) = each(%msgids) ) {
                    if ($line =~ /$key/) {
                        my @id = ($line =~ /\ ([0-9A-F]{11,12})/g);
                        $msgids{$key} = $id[1];
                    }
                }
            }
        }
        if ( $line =~ /to=<.*\.gov*.status=sent/ ) {
            if ( $line !~ /to=<.*(ourdomain\.gov|relay=localhost)/i) {
                while ( ($key, $value) = each(%msgids) ) {
                    if  ( defined $value ) {
                        if ($line =~ /$value/) {
                            print CLIENTOUT "$line\n";
                        }
                    }
                }
            }
        }
    }
    close FH;
}

[download]
  • Comment on Re: Structuring maillog data - hopefully simple question on arrays/hashes
  • Download Code

Replies are listed 'Best First'.
Re^2: Structuring maillog data - hopefully simple question on arrays/hashes
by McDarren (Abbot) on Jul 01, 2007 at 05:27 UTC
    (Moving this off my pad and into here - where it belongs)

    Try running the following code. I think it meets most of your requirements. To be perfectly honest, it's probably not much better than your existing code - but it does demonstrate a different approach, and the use of a HOH which I mentioned in my initial reply.

    If there are any parts of it that need explaining, feel free to ask :) 

    #!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;

# Replace the following with the location of your maillog
# (Don't forget to include the full path if it isn't in the cwd)
my $mail_log = 'billie_t.log';

my %messages;
my $valid_src = 'exch';
my $valid_dst = 'gov.au';
my $verbose = 1;    # Change this to 1 if you want to see which lines 
+are ignored

open my $mail_fh, '<', $mail_log
    or die "Cannot open $mail_log:$!\n";


while (my $line = <$mail_fh>) {
    chomp($line);
    if ($line =~ /qmgr.*?from=/) {      # It's a new mail
        my ($id, $from) = (split /\s+/, $line)[5,6];
        chop($id);                      # remove the colon
        $from =~ s/^from=<(.*?)>,/$1/;
        $messages{$id}{from} = $from;
    }
    elsif ($line =~ /client=/) {
        my ($id, $client) = (split /\s+/, $line)[5,6];
        chop($id);
        $client =~ s/client=(.*)/$1/;
        $messages{$id}{client} = $client;
    }
    elsif ($line =~ /relay=localhost/) {
        my ($id, $to, $status) = (split /\s+/, $line, 10)[5,6,9];
        chop($id);
        $to =~ s/to=<(.*?)>,/$1/;
        my ($remote_id) = $status =~ m/([0-9A-Z]+)\)$/;
        $status =~ s/.*?\((.*?)\)$/$1/;
        $messages{$id}{to} = $to;
        $messages{$id}{status} = $status;
        $messages{$id}{remote_id} = $remote_id;
    }
    else {
        print "Skipping... $line\n" if $verbose;
    }
}

for my $id (sort keys %messages) {
    no warnings 'uninitialized';
    next if (!$messages{$id}{client} || $messages{$id}{client} !~ /\Q$
+valid_src\E/);
    next if (!$messages{$id}{to} || $messages{$id}{to} !~ /\Q$valid_ds
+t\E/);
    print qq(Valid message found: ID=$id\n\tFROM=$messages{$id}{from}\
+n\t),
          qq(TO=$messages{$id}{to}\n\tSTATUS=$messages{$id}{status}\n\
+t),
          qq(CLIENT=$messages{$id}{client}\n\tREMOTE_ID=$messages{$id}
+{remote_id}\n\n);
}

# Uncomment the following line to get a better picture of the datastru
+cture that is created
# (You might want to redirect the output to a file)
#print Dumper(\%messages);

close $mail_fh;

    [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom