Re: Multiple patterns match in a big file and track the counts of each pattern matched

Instead of matching several times, create a larger regex and match just once.

In order to remember what part matched, I created a named capture groups and retrieved the names from the special %- hash. Without your data, I can't test whether it's faster or not.

#!/usr/bin/perl
use warnings;
use strict;

...

my @pat_array = split /@@@/, $InListOfPatterns;
my $i;
my $regex = join '|', map +($i++, "(?<m$i>$_)")[1], map quotemeta, @pa
+t_array;

open my $LOG,'<', $InLogFilePath or die "can not open file :$!";
my %matched;
while (<$LOG>) {
    chomp;
    if ( $. > $InStartLineNumber) {
        $matched{ (grep defined $-{$_}[0], keys %-)[0] }++ if /$regex/
+;
    }
}
close $LOG;

for my $pattern (keys %matched) {
    print "$pattern\t$matched{$pattern}\n";
}
[download]

Beware! You have a precedence issue in your code:

open LOG_READ,'<',"$InLogFilePath" || die "can not open file :$!";
[download]

This will never die if you can't open the file, it will only die if the file name is empty.

($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
[download]

Comment on Re: Multiple patterns match in a big file and track the counts of each pattern matched Select or Download Code

Replies are listed 'Best First'.
Re^2: Multiple patterns match in a big file and track the counts of each pattern matched by ansh007 (Novice) on Dec 04, 2017 at 11:07 UTC
I tried this. It works, but it takes more than 3 and a half minutes. I guess because of grep, but i am not sure. Thanks for your time.	[reply]
Re^3: Multiple patterns match in a big file and track the counts of each pattern matched by choroba (Cardinal) on Dec 04, 2017 at 11:41 UTC
Yes, that's probably the reason. What about embedded code? `my %matched; my $i; my $regex = join '\|', map +($i++, "$_(?{\$matched{$i}++})")[1], map qu +otemeta, @pat_array; open my $LOG,'<', $InLogFilePath or die "can not open file :$!"; while (<$LOG>) { use re 'eval'; chomp; /$regex/ if $. > $InStartLineNumber; } close $LOG; for my $pattern (keys %matched) { print "$pattern\t$matched{$pattern}\n"; }` [download] ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l] [select]