in reply to Parsing many files and output to one file. Pls HELP

I think the problem might be solved by the code below for any number of su and mu files. It doesn't check if every file has the same number of requests, (would need to add some test). Also, the glob function assumes all the files are in the current working directory. If that's not the case, then a path would need to be prepended to the glob expression. I used Sort::Naturally to sort the respective files saved in the arrays and List::Util for the sum function.

Chris

#!/usr/bin/perl use strict; use warnings; use Sort::Naturally; use List::Util 'sum'; use List::Compare; my %data; my %file_requests; my $base; my @su_files = nsort glob "su[1-9]*_sdkperfmonlog.txt"; my @mu_files = nsort glob "mu[1-9]*_sdkperfmonlog.txt"; for my $file (@su_files, @mu_files) { my $request; open my $fh, "<", $file or die "Unable to open $file for reading. +$!"; while (<$fh>) { if (/Start\s+-+\s+\d+:\d+:(.+)/) { $request = $1; } elsif (/ProcessRequest\t([.\d]+)/) { $data{$request}{$file} = $1; push @{ $file_requests{$file} }, $request; } } close $fh or die "Unable to close $file. $!"; # validate $base ||= $file_requests{$file}; my $lc = List::Compare->new( $base, $file_requests{$file} ); die "Non-matching requests from $file. $!" unless $lc->is_LequivalentR; } # Header print join("\t", 'Request Name', (map {/(su\d+)_/} @su_files), 'Su-Avg', (map {/(mu\d+)_/} @mu_files), 'Mu-Avg' ), "\n"; # Body for my $request (keys %data) { print $request, "\t"; my @vals = @{ $data{$request} }{ @su_files }; print join("\t", @vals, sprintf "%0.3f", sum(@vals)/@vals), "\t"; @vals = @{ $data{$request} }{ @mu_files }; print join("\t", @vals, sprintf "%0.3f", sum(@vals)/@vals),"\n"; }

Update: Added validation code

Replies are listed 'Best First'.
Re^2: Parsing many files and output to one file. Pls HELP
by hiradhu (Acolyte) on Nov 13, 2008 at 09:22 UTC
    Thanks a lot! I understood how u have used HASH & MAP to get this. MAP was the one I didnt think of using at all. I have added some more validations and regex to get the functionality I need. Thanks a lot!