I can think of a few things. First, read the header block all at once. That will reduce the number of reads to one per file.

Then anchor the regex on the beginning of a line. Specifically, only look on the To: line (I leave the Cc: line, as well as multiple addressees, as an excercise for later) and lose the /s modifier. The regex will fail immediately on any line that doesn't start with To:, only searching for email addresses on the rest of the To: line. This will reduce the number of matches to one per file.

Do a case-insensitive match, rather than lower casing the line. This will be one less operation per file.

#!/usr/bin/perl use strict; use warnings; use File::Find; use YAML::Syck; my %addresses; find(sub { return unless -f $_; open my $fh, '<', $_ or die; local $/ = ''; # "Paragraph" mode, reads a block of text to n +ext \n\n $_ = <$fh>; # Read Header block if (/To:.+\b(andrew\+[^\@\s]+\@[-a-z0-9\.]+)\b/mi) { # /m to ancho +r, /i to ignore case my $addr = $1; # some addresses are in mailing list bounce format if ($addr =~ s/[=\#\%](?:3d)?/@/) { # No need for /xms her +e $addr =~ s/\@[^@]+$//; } $addresses{$addr}++; } close $fh; }, glob($ENV{HOME} . '/Maildir/.misc*')); print Dump \%addresses;
You're using the /x, /m, and /s modifiers already, but I'm not sure you understand how to use them. /x allows you to put in whitespace for clarity, requiring you to use \s to match whitespace.
/To:.+ \b ( andrew\+ [^\@\s]+ \@ [-a-z0-9\.]+ ) \b /mix

The \s is fine in your original code, but you're not actually using any whitespace for clarity, so /x is unneeded.

Same with /s, where . matches newlines as if they were whitespace. For example, in the text below, /upon.+little/s would find a match,

Once upon a time\n
there was a little
prince\n
named Lord Fancypants.\n

while /upon.+little/ would not because . won't match a newline without /s.

Finally, /m allows you to match ^ and $ on a "line" -- that is, after and before a newline embedded in a block of text. So /^there/ would not find a match, while /^there/m would.

Once upon a time\n
there was a little prince\n
named Lord Fancypants.\n

And /^there.+/m would match "there was a little prince", while /^there.+/ms would match

Once upon a time\n
there was a little prince\n
named Lord Fancypants.\n

Similarly, in your substitution,  $addr =~ s/[=\#\%](?:3d)?/@/xms, you're not using whitespace for clarity, nor are you anchoring on the beginning or end of a line, nor should your match ever go over a newline. I'm not sure if it would make a performance difference, but you could just drop the /xms entirely, as I did above.

I hope this helps.

--marmot

In reply to Re: Looking for ideas on how to optimize this specialized grep by furry_marmot
in thread Looking for ideas on how to optimize this specialized grep by afresh1

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.