I can think of a few things. First, read the header block all at once. That will reduce the number of reads to one per file.
Then anchor the regex on the beginning of a line. Specifically, only look on the To: line (I leave the Cc: line, as well as multiple addressees, as an excercise for later) and lose the /s modifier. The regex will fail immediately on any line that doesn't start with To:, only searching for email addresses on the rest of the To: line. This will reduce the number of matches to one per file.
Do a case-insensitive match, rather than lower casing the line. This will be one less operation per file.
You're using the /x, /m, and /s modifiers already, but I'm not sure you understand how to use them. /x allows you to put in whitespace for clarity, requiring you to use \s to match whitespace.#!/usr/bin/perl use strict; use warnings; use File::Find; use YAML::Syck; my %addresses; find(sub { return unless -f $_; open my $fh, '<', $_ or die; local $/ = ''; # "Paragraph" mode, reads a block of text to n +ext \n\n $_ = <$fh>; # Read Header block if (/To:.+\b(andrew\+[^\@\s]+\@[-a-z0-9\.]+)\b/mi) { # /m to ancho +r, /i to ignore case my $addr = $1; # some addresses are in mailing list bounce format if ($addr =~ s/[=\#\%](?:3d)?/@/) { # No need for /xms her +e $addr =~ s/\@[^@]+$//; } $addresses{$addr}++; } close $fh; }, glob($ENV{HOME} . '/Maildir/.misc*')); print Dump \%addresses;
/To:.+ \b ( andrew\+ [^\@\s]+ \@ [-a-z0-9\.]+ ) \b /mix
The \s is fine in your original code, but you're not actually using any whitespace for clarity, so /x is unneeded.
Same with /s, where . matches newlines as if they were whitespace. For example, in the text below, /upon.+little/s would find a match,
Once upon a time\n
there was a little prince\n
named Lord Fancypants.\n
while /upon.+little/ would not because . won't match a newline without /s.
Finally, /m allows you to match ^ and $ on a "line" -- that is, after and before a newline embedded in a block of text. So /^there/ would not find a match, while /^there/m would.
Once upon a time\n
there was a little prince\n
named Lord Fancypants.\n
Once upon a time\n
there was a little prince\n
named Lord Fancypants.\n
I hope this helps.
--marmotIn reply to Re: Looking for ideas on how to optimize this specialized grep
by furry_marmot
in thread Looking for ideas on how to optimize this specialized grep
by afresh1
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |