in reply to Re^3: Parsing logs and bookmarking last line parsed
in thread Parsing logs and bookmarking last line parsed

@Marshall - I'm looking at your parse script you provided. Can you show me in the regex, how to get only specific attributes? In other words, I'm not interested in splitting all of the data, but interested in these keys for example (backup-set, backup-date, backup-time, ERROR) with the flexibility to add or remove more if needed.

Also can you show me how to how to print the keys out in order with a delimited format for example (name=value;name=value;) Thank you again.

  • Comment on Re^4: Parsing logs and bookmarking last line parsed

Replies are listed 'Best First'.
Re^5: Parsing logs and bookmarking last line parsed
by Marshall (Canon) on Aug 21, 2010 at 21:23 UTC
    I wouldn't fiddle around with the parse_line() subroutine. It has a job which is to "parse lines". You don't want to modify it so that it only parses "some of the lines". Try to write code so that each function has a single clear task.

    If you want a subset of the data then either change the code immediately after the line parsing (it only saves things which are of the form p=v now). Or filter what you need at the final "print it" stage, which is what I did below in some more code for you. I decided to do it that way so that when problems happen latter, you have all the p=v pairs and you can use Dumper to see what is going on. There is no need here to optimize to reduce memory storage.

    What to print is specified by an array (important_keys) and you can modify that as needed. If you want a special sort order, then modify the code I gave you earlier. BUT I would recommend against that in favor of a simple normal alphabetic sort. In my experience this usually works out best for "human readable" reports - humans scan sorted lists easily. Note: the hash important_order isn't used for a sort, but is used as an easy way to determine if some parm is in the "desired" list or not.

    You've got some work to do, but I think you are on the right path.

    As an update, to avoid sorting the final parms altogether, iterate through the important_keys array and print that key,value pair if it exists. The order in the array determines the "sort order" automatically (i.e. no sorting at all). Of course you could just sort the important_keys array in method 2 to get same as method 1, but you should know how to generically just print all the keys in this type of hash structure, so I left method 1 in.

    #!/usr/bin/perl -w use strict; use Data::Dumper; my %backups; my @important_keys = qw(backup-status backup-set backup-date backup-ti +me ERROR); # this is how to translate the array into something that could # be used for a special sort order, but I don't think you will # need to do that. my $i=1; my %important_order = map{$_ => $i++} @important_keys; while (<DATA>) { next if (/^\s*$/); #skip blank lines chomp; my ($date, $backupset , $parm , $value) = parseline($_); if ($value) { $backups{$backupset}{$parm} = $value; } } #print Dumper \%backups; foreach my $set (sort keys %backups) { print "\n*** backup set: $set ***\n"; #method 1 foreach my $parm (sort keys %{$backups{$set}}) { printf " %-15s = %s \n", $parm, $backups{$set}{$parm} if $important_order{$parm}; } #method 2 # foreach my $parm (@important_keys) # { # printf " %-15s = %s \n", $parm, $backups{$set}{$parm} # if $backups{$set}{$parm}; # } } sub parseline { my $line = shift; my ($date, $rest) = $line =~ m/(^.*\d{4}):(.*)/; my ($backupset, $msg) = split(/backup:INFO:/, $rest); $backupset =~ s/:\s*$//; #trimming some unwanted thing like ':' is + ok $backupset =~ s/^\s*backup\.//; #more than one step is just fine to +o! my ($parm, $value) = $msg =~ m/\s*(.*)=\s*(.*)\s*/; $parm ||= $msg; #if match doesn't happen these will be undef $value ||=""; #so this trick makes sure that they are defined. return ($date, $backupset, $parm, $value); } =method 1 prints: *** backup set: set1_lvm *** backup-date = 20100816000003 backup-set = backup.set1_lvm backup-status = Backup succeeded backup-time = 01:59:04 *** backup set: set2_lvm *** backup-date = 20100815200003 backup-set = backup.set2_lvm *** backup set: set2_lvm_lvm *** backup-status = Backup succeeded backup-time = 04:33:12 =method 2 prints: *** backup set: set1_lvm *** backup-status = Backup succeeded backup-set = backup.set1_lvm backup-date = 20100816000003 backup-time = 01:59:04 *** backup set: set2_lvm *** backup-set = backup.set2_lvm backup-date = 20100815200003 *** backup set: set2_lvm_lvm *** backup-status = Backup succeeded backup-time = 04:33:12 =cut
    data_segment follows (its same as previous posts)