Hi Monks,

I have a question about a script that loops one list to look for values in another list. I am happy to say the script produces the correct results, but I am surprised at how long it takes. The first list is 72,000 lines long and it is searching through another list that is only 200 lines lines long. To process a 1.2MB list with a 102KB list, takes about 23 minutes. If that’s how long it takes that’s ok. But I have a strong suspicion that I entered some faulty logic here.

This is my code that works but runs slow:

use strict;
use warnings;
…..

#Array constains 200 elements, each element is a name and a title tab 
+separated
@flab = [ ‘Name’ and a ’Title’ ]

#open LONG_LIST of 72,000 lines for reading line by line
open (LONG_LIST, $longList) or die "Cannot open file"; 


 print RESULT “Company"."\t”.”Name"."\t”.”Title"."\n"; 

 while (my $sLine = <LONG_LIST>){    
     chomp $sLine; 
     #split LONG_LIST lines into two fields ‘Company' and ’Name'
     my @sTab = split (/\t/,$sLine); 
     
     foreach (@flab){ 
           my $line = $_;
               
           #split each @flab line into two fields ‘Name’ and ’Title'
           my @match = split (/\t/,$line); 

            #match ’Name’ in LONG_LIST to ‘Name' in @flab
           if (($match[0] =~/\Q$sTab[1]\E/) and (define $match[1])){ 
            #print ‘Company', ‘Name', ’Title' 
             print RESULT $sTab[0]."\t".$sTab[1]."\t".$match[1]."\n";
           } 
       }
 }

…..

__END__

[download]

No issues with warnings or strict compilation errors and as I mentioned it works. It produces a list of all the names with their companies and their title. It just takes really long. I suspect my logic is doing an unnecessary loop or processing that delays completion but I don’t know what I can change. Any feedback or tips on restructuring my code would be most appreciated.

Thank you!

In reply to Script Executes Very Slowly by doubleqq

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.