rockyurock has asked for the wisdom of the Perl Monks concerning the following question:

In my below program , I want to search all the keywords provided In text file named - Success_MessageFlow.txt from a list of file named with *main_log and I am getting the output In each different file named Message_Flow.$l.txt according to the no of files present In a directory My keyword file sample contains :
Hello Sample Hello Sample#Step-9:Hello# Bye Sample Bye Sample#Step-3:Bye#
My Input file content sample Is :
12-26 13:35:06.057570 236 578 D Hello Sample Hello Sample 12-26 13:35:05.623529 236 578 D Bye Sample Bye Sample
My output generated Is :
*******#Step-9:Hello Sample Hello Sample#******* Hello Sample Hello Sample in file main_log, line 1:12-26 13:35:06.0575 +70 *******#Step-3:Bye Sample Bye Sample#******* Bye Sample Bye Sample in file main_log, line 1:12-26 13:35:05.623529
Issue : Though both Step-9 and Step-3 are searched correctly but I want searching to be In order of timing or Step-3 should be printed In my output file before Step-9 as step-3 keyword occurrence was before (12:35:05) than the Step-9 (12:35:06) Can anyone please help to suggest any modification in my program that can help ? My Program :
printf "\n Starting success message flow\n\n"; # Opening Keyword File here open( my $kw, '<', 'Success_MessageFlow.txt') or die $!; my @keywords = <$kw>; sort @keywords; chomp(@keywords); # remove newlines at the end of keywords # post-processing your keywords file for adding comments my $kwhashref = { map { /^(.*?)(#.*?#)*$/; defined($2) ? ($1 => $2) : ( $1 => undef ) } @keywords }; # get list of files in current directory my @files = <main_log*>; my $l = 0; # loop over each file to search keywords in foreach my $file (@files) { print "\n Processing with file $file \n"; open(my $fh, '<', $file) or die $!; my @content = <$fh>; sort @content; close($fh); $l++; open my $out_file, '>', "Message_Flow.$l.txt" or die "$!"; foreach my $kw (keys %$kwhashref) { my $search = quotemeta($kw); # otherwise keyword is used as re +gex, not literally foreach (@content) { # go through every line for this keyword if (/$search/) { printf $out_file "\n*******$kwhashref->{$kw}** +*****\n"."\n" if defined($kwhashref->{$kw}); printf $out_file '%s in file %s, line %d:%s'.$ +/, $kw, $file, $l, $_; } } } } printf "Check the output generated In file IMS_Reg_Message_Flow.txt\n" +;

Replies are listed 'Best First'.
Re: How to search keywords In sequence or according to timing from a text files In Perl ?
by poj (Abbot) on Feb 22, 2017 at 12:27 UTC

    Push the matching records to an array and sort them before outputting. For example

    #!perl use strict; use Data::Dump 'pp'; # build key words hash print "\n Starting success message flow\n\n"; my $kwhashref = get_keywords('Success_MessageFlow.txt'); #pp $kwhashref; # create regex my $match = join '|',map{quotemeta} keys %$kwhashref; my $re = qr/$match/; #print $re; # get list of files in current directory my @files = <main_log*>; # process records #12-26 13:35:05.623529 236 578 D Bye Sample Bye Sample my @output = (); foreach my $file (@files) { print "\n Processing with file $file \n"; open my $fh, '<', $file or die $!; my $count = 0; while (<$fh>){ ++$count; # match any keyword if (/($re)/){ my $kw = $1; push @output, [$kw,$file,$count,$_]; } } close $fh; } # sort by record timestamp @output = sort { $a->[3] cmp $b->[3]} @output; #pp \@output; # output my $outfile = 'IMS_Reg_Message_Flow.txt'; print "Check the output generated In file $outfile\n"; open my $fh_out, '>', $outfile or die "$!"; for (@output){ my $kw = $_->[0]; if ( defined $kwhashref->{$kw} ){ print $fh_out "\n*******$kwhashref->{$kw}*******\n"; } printf $fh_out '%s in file %s, line %d:%s'.$/, @$_; } close $fh_out; # get keywords sub get_keywords { my $infile = shift; my %keyword=(); open( my $in,'<', $infile) or die $!; while (<$in>){ if (/^(.*?)(#.*?#)*$/){ $keyword{$1} = $2; } } close $in; return \%keyword; }
    poj
      Hi,, Can I get the output Into respective file rather than Into single file e.g. If I am parsing keywords from main_log then output something like output_main_log & if I am parsing though main_log(2) then output generated In output_main_log(2)rather than In the single file to have more clarity ? Thanks

        Move the output code inside the @files loop

        foreach my $file (@files) { my @output = (); print "\n Processing with file $file \n"; open my $fh, '<', $file or die $!; my $count = 0; while (<$fh>){ ++$count; # match any keyword if (/($re)/){ my $kw = $1; push @output, [$kw,$file,$count,$_]; } } close $fh; # sort by record timestamp @output = sort { $a->[3] cmp $b->[3]} @output; #pp \@output; # output my $outfile = 'output_'.$file; print "Check the output generated In file $outfile\n"; open my $fh_out, '>', $outfile or die "$!"; for (@output){ my $kw = $_->[0]; if ( defined $kwhashref->{$kw} ){ print $fh_out "\n*******$kwhashref->{$kw}*******\n"; } printf $fh_out '%s in file %s, line %d:%s'.$/, @$_; } close $fh_out; }
        poj
Re: How to search keywords In sequence or according to timing from a text files In Perl ?
by 1nickt (Canon) on Feb 22, 2017 at 11:16 UTC

    You could build up another hash with the matches found, and sort by the timestamp before printing to your output file. See sort for documentation on how to create a custom sort subroutine.

    Hope this helps!


    The way forward always starts with a minimal test.
Re: How to search keywords In sequence or according to timing from a text files In Perl ?
by BillKSmith (Monsignor) on Feb 22, 2017 at 16:19 UTC
    Cross posted on http://perlguru.com/gforum.cgi?post=83798. We have no objection to posting on other forums, but please tell us where else you have posted the same question so we can avoid duplicate effort.
    Bill