Optimization is of course a science and it should be done by locating hotspots with a profiler (Devel::NYTProf), and attacking the places where the code is spending the most time <update2> and using a module like Benchmark to test and compare alternatives </update2>. But here are some rough rules of thumb / unverified gut feelings based on experience (every mention of "slow" below is therefore subjective):

• grep will loop over the entire array, so running it multiple times will loop over the list multiple times, and will therefore be slow
• you are right that eval on a string is relatively slow
• (any_list_context) = <$FILE>, as grep and foreach require, will load the entire file into memory (slow and memory-hungry)
• therefore, foreach (@lines) will be slower than a while (<>) loop, which reads and processes only one line at a time
• instead of your while (<>) { foreach my $pattern (@patterns) { ... } } (nested loops always slow things down), I would suggest a single regex, or regexen combined with logic (if (/regex1/ && /regex2/))

So personally my starting point would be something like this, based on Building Regex Alternations Dynamically:

my @strings_to_match = ('192.168.200.', '10.10.200');
my ($regex) = map { qr/$_/ }
    join '|', map {quotemeta}
    sort { length $b <=> length $a }
    @strings_to_match;

my @filtered_result;
while (<>) {
    push @filtered_result, $_
        if m{ ,SEVERE, .* $regex }x;
}
[download]

Note that I am only recommending this solution because you said "it didn't really matter to me where the IP address I was filtering was located, (just that it was there)". Otherwise, I would recommend Text::CSV_XS for loading CSV files.

I was hoping to find a solution, however, where the grep function could be expanded (nested?) an arbitrary number of times based on the different match strings that we get.

No, sorry, it doesn't work that way*, however, you can make the matching logic for a single loop (while/foreach/grep) as complex as you need, as I showed above. I would not recommend building a string of Perl code and evaling it either, because it's much too easy to get burned (and in some cases expose security holes).

Minor edits for clarity.

* Update: The solution would be "lazy lists", which is something that can be implemented by iterators - although this is probably much too advanced for now (and probably wouldn't give you a performance gain either), Dominus's book Higher-Order Perl is a wonderful read on that topic.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.