I've done some benchmarking of the three options (not including lima1's suggestion of suffix-trees): sequential regular expressions, sequential index's, and merging the regular expressions together using Regexp::Assemble. I'm not great at writing benchmarks, so perhaps this script is simplistic, and I'd appreciate comments on how to improve it. Nevertheless, here are the results:

Regexp : 230 wallclock secs (229.86 usr +  0.06 sys = 229.92 CPU)
Index  : 22 wallclock secs (21.47 usr +  0.01 sys = 21.48 CPU)
Merged : 663 wallclock secs (663.73 usr +  0.17 sys = 663.90 CPU)

[download]

So it appears that even without forking the indexing approach is much faster for exact matches. Here's the benchmarking code I used.

use strict;
use Bio::SeqIO;
use Benchmark;
use Regexp::Assemble;
use Carp;

my $seqio = Bio::SeqIO->new(
        -file => "chrY.fa.masked",
        -format => "fasta"
        );
my $seq = $seqio->next_seq();
my $sequence = $seq->seq();

my @strings = (
        'CACGTG',
        'GTGCAC'
        );

my $t0 = new Benchmark;
for (1..100) {
        foreach my $string (@strings) {
                my $len = length($string);
                while ($sequence =~ /$string/g) { my $val = pos($seque
+nce) - $len; }
                }
        }
my $t1 = new Benchmark;

my $t2 = new Benchmark;
for (1..100) {
        foreach my $string (@strings) {
                my $pos = 0;
                my $len = length($string);
                while ( ($pos = index($sequence, $string, $pos)) >= 0 
+) {
                        my $val = substr($sequence, $pos, $len);
                        $pos += $len;
                        }
                }
        }
my $t3 = new Benchmark;

my $t4 = new Benchmark;
for (1..100) {
        my $ra = Regexp::Assemble->new();
        $ra->add(@strings);
        while ( $sequence =~ /$ra/g) { my $val = pos($sequence) - 6; }
        }
my $t5 = new Benchmark;

my $td1 = timediff($t1, $t0);
my $td2 = timediff($t3, $t2);
my $td3 = timediff($t5, $t4);

print 'Regexp : ', timestr($td1), "\n";
print 'Index  : ', timestr($td2), "\n";
print 'Merged : ', timestr($td3), "\n";

[download]

In reply to Re: Multiple Regex's on a Big Sequence - Benchmark by bernanke01
in thread Multiple Regex's on a Big Sequence by bernanke01

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.