(the previous thread he mentions is "Pattern Matching", not using regex)

How many strings are you searching for? Is it just 2 or 3, or is it 200-300?

Can the match span lines in the big file? Like this, for instance: 

...............AAAAAAAA"ATGGCTC
GTGTCCA"AAAAAAAAAAA ...........

[download]
That would obviously complicate things...

If you only have a manageable number of strings to match, and a match can't occur across lines, I'd suggest something like this (which does use regexes, but if you have multiple strings to match at once, I don't know how to avoid them easily): 

use Regex::PreSuf;

sub superMatch {
    my ($patternFile, $dataFile, $outFile)=@_;
    open(PAT,"<$patternFile") or die "Can't open $patternFile, error $
+!";
    my @patterns=<PAT>;
    chomp @patterns;
    close(PAT);

    open(OUT, ">>$outFile") or die "Can't open output file $outFile, e
+rror $!";

    # Regex::PreSuf generates a regex that will match all
    # of the patterns much more quickly than a naive 
    # join "|",@patterns will
    my $re=presuf(@patterns);

    open(DATA,"<$dataFile") or die "Can't open $dataFile, error $!";
    # NO NEED TO READ INTO MEMORY ALL AT ONCE!
    while(<DATA>) {
        # only compile regex once
        if(/$re/o) {
           # only chomp if we have a match
           chomp;
           # capturing matches are slower, so only capture 
           # if one or more matches are present.
           # might have more than one match in a line!
           while(/($re)/og) {
               print OUT "'$1', '$_'";
           }
        }
    }
}

[download]
I haven't tested this, but it could be a start for you...
--
Mike

In reply to Re: Quickest method for matching by RMGir
in thread Quickest method for matching by dr_jgbn

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.