in reply to Re^4: Parsing logs and bookmarking last line parsed
in thread Parsing logs and bookmarking last line parsed

I wouldn't fiddle around with the parse_line() subroutine. It has a job which is to "parse lines". You don't want to modify it so that it only parses "some of the lines". Try to write code so that each function has a single clear task.

If you want a subset of the data then either change the code immediately after the line parsing (it only saves things which are of the form p=v now). Or filter what you need at the final "print it" stage, which is what I did below in some more code for you. I decided to do it that way so that when problems happen latter, you have all the p=v pairs and you can use Dumper to see what is going on. There is no need here to optimize to reduce memory storage.

What to print is specified by an array (important_keys) and you can modify that as needed. If you want a special sort order, then modify the code I gave you earlier. BUT I would recommend against that in favor of a simple normal alphabetic sort. In my experience this usually works out best for "human readable" reports - humans scan sorted lists easily. Note: the hash important_order isn't used for a sort, but is used as an easy way to determine if some parm is in the "desired" list or not.

You've got some work to do, but I think you are on the right path.

As an update, to avoid sorting the final parms altogether, iterate through the important_keys array and print that key,value pair if it exists. The order in the array determines the "sort order" automatically (i.e. no sorting at all). Of course you could just sort the important_keys array in method 2 to get same as method 1, but you should know how to generically just print all the keys in this type of hash structure, so I left method 1 in.

#!/usr/bin/perl -w use strict; use Data::Dumper; my %backups; my @important_keys = qw(backup-status backup-set backup-date backup-ti +me ERROR); # this is how to translate the array into something that could # be used for a special sort order, but I don't think you will # need to do that. my $i=1; my %important_order = map{$_ => $i++} @important_keys; while (<DATA>) { next if (/^\s*$/); #skip blank lines chomp; my ($date, $backupset , $parm , $value) = parseline($_); if ($value) { $backups{$backupset}{$parm} = $value; } } #print Dumper \%backups; foreach my $set (sort keys %backups) { print "\n*** backup set: $set ***\n"; #method 1 foreach my $parm (sort keys %{$backups{$set}}) { printf " %-15s = %s \n", $parm, $backups{$set}{$parm} if $important_order{$parm}; } #method 2 # foreach my $parm (@important_keys) # { # printf " %-15s = %s \n", $parm, $backups{$set}{$parm} # if $backups{$set}{$parm}; # } } sub parseline { my $line = shift; my ($date, $rest) = $line =~ m/(^.*\d{4}):(.*)/; my ($backupset, $msg) = split(/backup:INFO:/, $rest); $backupset =~ s/:\s*$//; #trimming some unwanted thing like ':' is + ok $backupset =~ s/^\s*backup\.//; #more than one step is just fine to +o! my ($parm, $value) = $msg =~ m/\s*(.*)=\s*(.*)\s*/; $parm ||= $msg; #if match doesn't happen these will be undef $value ||=""; #so this trick makes sure that they are defined. return ($date, $backupset, $parm, $value); } =method 1 prints: *** backup set: set1_lvm *** backup-date = 20100816000003 backup-set = backup.set1_lvm backup-status = Backup succeeded backup-time = 01:59:04 *** backup set: set2_lvm *** backup-date = 20100815200003 backup-set = backup.set2_lvm *** backup set: set2_lvm_lvm *** backup-status = Backup succeeded backup-time = 04:33:12 =method 2 prints: *** backup set: set1_lvm *** backup-status = Backup succeeded backup-set = backup.set1_lvm backup-date = 20100816000003 backup-time = 01:59:04 *** backup set: set2_lvm *** backup-set = backup.set2_lvm backup-date = 20100815200003 *** backup set: set2_lvm_lvm *** backup-status = Backup succeeded backup-time = 04:33:12 =cut
data_segment follows (its same as previous posts)
__DATA__ Sun Aug 15 20:00:03 2010: backup.set2_lvm:backup:INFO: START OF BACKUP Sun Aug 15 20:00:04 2010: backup.set2_lvm:backup:INFO: backup-set=back +up.set2_lvm Sun Aug 15 20:00:04 2010: backup.set2_lvm:backup:INFO: backup-date=201 +00815200003 Sun Aug 15 20:00:04 2010: backup.set2_lvm:backup:INFO: backup-type=reg +ular Sun Aug 15 20:00:04 2010: backup.set2_lvm:backup:INFO: backup-date-epo +ch=1281927603 Sun Aug 15 20:00:04 2010: backup.set2_lvm:backup:INFO: backup-director +y=/home/backups/backup.set2_lvm/20100815200003 Mon Aug 16 00:00:04 2010: backup.set1_lvm:backup:INFO: START OF BACKUP Mon Aug 16 00:00:05 2010: backup.set1_lvm:backup:INFO: backup-set=back +up.set1_lvm Mon Aug 16 00:00:05 2010: backup.set1_lvm:backup:INFO: backup-date=201 +00816000003 Mon Aug 16 00:00:05 2010: backup.set1_lvm:backup:INFO: backup-type=reg +ular Mon Aug 16 00:00:05 2010: backup.set1_lvm:backup:INFO: backup-date-epo +ch=1281942003 Mon Aug 16 00:33:15 2010: backup.set2_lvm_lvm:backup:INFO: last-backup +=/home/backups/backup.set2_lvm_lvm/20100814200003 Mon Aug 16 00:33:15 2010: backup.set2_lvm_lvm:backup:INFO: backup-size +=424.53 GB Mon Aug 16 00:33:15 2010: backup.set2_lvm_lvm:backup:INFO: backup-time +=04:33:12 Mon Aug 16 00:33:15 2010: backup.set2_lvm_lvm:backup:INFO: backup-stat +us=Backup succeeded Mon Aug 16 00:33:15 2010: backup.set2_lvm_lvm:backup:INFO: Backup succ +eeded Mon Aug 16 00:33:16 2010: backup.set2_lvm_lvm:backup:INFO: END OF BACK +UP Mon Aug 16 01:59:07 2010: backup.set1_lvm:backup:INFO: last-backup=/ho +me/backups/backup.set1_lvm/20100815000006 Mon Aug 16 01:59:07 2010: backup.set1_lvm:backup:INFO: backup-size=187 +.24 GB Mon Aug 16 01:59:07 2010: backup.set1_lvm:backup:INFO: backup-time=01: +59:04 Mon Aug 16 01:59:07 2010: backup.set1_lvm:backup:INFO: backup-status=B +ackup succeeded Mon Aug 16 01:59:07 2010: backup.set1_lvm:backup:INFO: Backup succeede +d Mon Aug 16 01:59:09 2010: backup.set1_lvm:backup:INFO: END OF BACKUP