in reply to Re^2: How to split file for threading?
in thread How to split file for threading?

If you can search the whole line without breaking it up, this:

for my $line (<INFILE>) { chomp $line; my @splitline = split("\t", $line); #my $poskey = $splitline[0] . ":" . $splitline[1]; for (@splitline){ if (/^$ARGV[1]/){ $filter_count++; push(@rawfile,$line); } } }

can become this:

while (my $line = <INFILE>) { chomp $line; if ($line =~ /^$ARGV[1]/){ $filter_count++; push(@rawfile,$line); } }

-stevieb

Replies are listed 'Best First'.
Re^4: How to split file for threading?
by afoken (Chancellor) on Jun 23, 2015 at 16:40 UTC
    while (my $line = <INFILE>) { chomp $line; if ($line =~ /^$ARGV[1]/){ $filter_count++; push(@rawfile,$line); } }

    A little bit more fine-tuning:

    • If the line does not match, there is no point in chomping it. So move chomp into the if block. (This assumes the search pattern has no problems with the trailing line-end character(s) in $line.)
    • Using a variable inside the regexp forces perl to recompile it several times (I think). my $pattern=qr/^$ARGV[1]/; in front of the while loop and using $line=~$pattern in if would avoid that.
    • If the search is for a keyword at the first character of the line, and not a pattern, using a regexp may be slower than a simple string operation. Try using if (index($line,$ARGV[1])==0) or if (substr($line,0,length($ARGV[1])) eq $ARGV[1]) instead.

    Hint: Use Devel::NYTProf to locate hot spots in your code.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re^4: How to split file for threading?
by diyaz (Beadle) on Jun 23, 2015 at 15:52 UTC
    thank you!