Looking at the second example code, it looks like the regexps are being built from scratch on a per use basis, in spite of there being only one call to build them all at the beginning.

The first example seems therefore a better basis to optimise from. And I would be inclined to blame its performance problems on the fact that you are getting a new output filehandle per line which is causing huge object proliferation to take place. You only need the output filehandle to be constructed once, e.g.: 

my %lookingFor; 
# keys => different name of one subset
# values => array of one subset
my $fh = new FileHandle "< largeLogFile.log";
my $writeFh = new FileHandle ">> myout.log";

while (<$fh>) {
  foreach my $subset (keys %lookingFor) {
    foreach my $item (@{$subset}) {
      if (<$fh> =~ m/$item/) {
        print $writeFh <$fh>;
      }
   }
}
close $fh or die $!;
close $writeFh or die $!;

[download]
Update: your code will also fail a "use strict" which should be placed at the beginning. To solve that, construct the hash so that its values are array references rather than array names and the inner loop should change to...
    foreach my $item ( @{$lookingFor{ $subset }} {

[download]

-M

Free your mind

In reply to Re: how to parse large files by Moron
in thread how to parse large files by domcyrus

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.