hello, I a noob at perl, I wrote a small script to do the following: 1.) read log search for keyword error 2.) read error waiver file. 3.) decide log passed/failed. this is the main section of the code 
%g_wvr_list = ();
%g_log_err = ();
@tmpArr = ();

#get errors from waiver list
open(IFP0,"<$opt_wvr_file");
@tmpArr=<IFP0>;
foreach $el (@tmpArr){
  chomp($el);
  $g_wvr_list{"$el"}=1;
}
close(IFP0);

#get errors from test log file
@tmpArr=`grep -iw error $opt_test_log`;
$errorCnt=$#tmpArr+1;

#pass /fail
$waived=0;
printDbg(@tmpArr);
foreach $key ( keys(%g_wvr_list) ){
  printDbg("wvr List $key");
  @matchErr = grep( /\Q$key\E/ , @tmpArr );
  printDbg("matchErrCnt {$#matchErr}");
  printDbg("matchErrCnt {@matchErr}");
  $waived+= @matchErr if( $#matchErr + 1 > 0 );
}

[download]
i wish to optimize this code specially the cross checking section. I will be handling log in sizes of 100 MB and error lines of 1000s . and my approach is slow. any idea on improvement is highly appreciated. thank you

In reply to efficient text processing by sarutobi

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.