Hello All,
I currently have 2 large files, each about 65,000 lines each. The files include lines of data, all seperated by "pipes". The first 3 fields of the files are what differentiats (sp?) each line for every cell sight in the nextel system, per half hour for the last 24 hours. So, for exmaple, the first line would look like: 
20035pa|02/06/2001|8:30| from here on is about 60 more fields of data

[download]

In any case, i have a program which searches this file, but it runs extremely slow, I believe it happens because once it finds the search string, it still continues to look through the entire script for the search string. However, the files only contain one combination for each site/per date/per half our. Here is the relevant code:
foreach $item(@timearray){
    
    $searchstring = "$NETID\|$month\/$date\/$year\|$item\|";
        chomp($ECL=`grep "^$searchstring" /PHL/data1/PHL/tmp/ECL_STAT.
+txt`);
        if ( $ECL == "" ) { push @ECL, "||$item||"; }
        else { push @ECL, $ECL;}
}

[download]
The variables are: 
$NETID = the network id, which would correspond to 20035pa in the abov
+e example.
$month, $date, $year = would correspond to 02/06/2001 in the above exa
+mple.
$item = the time, which would be 8:30 above.

[download]
I am using unix grep in the program, my question is, would using perl grep speed the process? Any tips on speeding it up?
Here are some solutions i came up with, im not sure how effective they are, if possible:
1: Start the grep where it left off, with the next search string, becasue the files are sorted by half hour. (is that possible to do?) 2: Terminate the grep once string is found, then pick up for the next string (in which the $item would have changed)

I dont prefer unix grep over perl grep, im just looking for ways to speed up the program.

Thanks In Advance


Dipul

In reply to Grep Speeds by ImpalaSS

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.