my @array;
my $match = 'some_criteria';

while (<FH>) {
    chomp;  # thanks gwadej for reminding me
    my ($x,$y) = split /\t/, $_;
    if ( $x eq $match ){   
                push @array, $y;
             }
}
 
print "@array\n";
[download]

You might want to add a chomp before the split.

G. Wade

thanks, that's indeed a neat way to do it!

If you want to know why it's so slow, profile it. Until you do that, you're not only wasting your time trying to speed up things that may not be slow, you're also wasting your time guessing. There's just no reason to sit and scratch your head when you can sit and watch the computer tell you the answer.

thanks I will do that.

I was almost with this up to the regex /^(.*)\t(.*)$/. Can you post the working code, from which one can unambiguously reverse-engineer the requirement ?

When you say you are searching for a "given string", what do you mean ? Are you looking for lines in the file where the "block" before the tab exactly matches the given string ? or the given string as a sub-string of one, or either or both blocks ? or what ?

I will post some code asap. I'm indeed looking for a substring in the left block. I'm using m=~ to achieve this.

OK. From what I've understood you to want... see if something along these lines works any better:

use strict ; use warnings ;

my $data ;
{ local $/ ; $data = <DATA> ; } ;

my $s = 'bc' ;  # Substring to search for in left hand side

my @r = ($data =~ m/$s[^\s]*[ \t]+(.*)/g) ;
print "@r\n" ;

__DATA__
abcd    s1
efgh    s2
ijklm    s3
nopq    s4
bc    s5
aqbc    s6
rst    s7
uvwx    s8
yz    s9
[download]

noting that the [ \t]+ can be changed to simply \t for use with your file (I've used [ \t]+ here only because that works even if the stuff between the "blocks" has been converted to any mixture tabs and spaces). Noting also that the (.*) will stop at end of line.

Note also that this is assuming that the right hand side will not contain any tabs (or spaces)

The regex can match to the search substring in the right hand side, but since that ends in a newline, the [^\s]*[ \t]+ part will fail the match.

The following:

my $data = "\n" ;
{ local $/ ; $data .= <DATA> ; } ;

my $s = 'bc' ;  # Substring to search for in left hand side

my @r = ($data =~ m/\n.*?$s[^\s]*[ \t]+(.*)/g) ;
[download]

avoids matching in the right hand side, and the right hand side can contain tabs (and spaces). I don't have enough data to know if there's any performance advantage or disadvantage here.

I'm using m=~ to achieve this.

I guess that this is a typo—if you've got working code at all, then you must be using something like (if we must call it that) =~m//, not m=~.

Think some more about this file.

Is the file only rarely changed? Or, is it changed much more rarely than it is searched? Or, at least, are the changes, when they come, relatively small?

If so, you could benefit enormously by importing the file into a database once, and then letting the database do the heavy lifting of indexing and searching.

Well the data is rarely changed - it is generated once every month. I will think about the database solution.

I use the data to generate ngrams. Basically, whenever I find a given string in the first part of the line (before the \t), I put the second into an array. After having gone through all the lines, I generate ngrams for the array.

Is this something where using DBM::Deep to store the resulting data structure for reader use would make sense?

--MidLifeXis

I don't believe reading the file to an array would do much else than increase the memory usage of your script. Most time is probably spent reading from disk anyways.

Don't quote me on this, but you could probably gain some speed boost by replacing the regexp with split(/\t/, $_);. Other than that, I'm out of ideas..

It may not just be the regular expression that is causing your perceived performance issues.

Can you show some code to illustrate what you have tried?