Hello and thanks in advance

I'm a perl novice and have some trouble with the following script being inefficient. I have a data input file containing sequences and their positions in the human genome, with each line as a separate sequence. I want to compare the positions of each sequence to a reference file and extract the name of the gene, which is the last column in the reference file. Both files are tab delimited and have a lot of extraneous information. The script goes line by line from the input and for each line searches criteria in the reference line by line. My script works, but as both of these files are large, this can mean runtime of several days. Is there any way to hash one of the files, so that it only iterates N times, vs N*M times, where N and M are line counts for each file? Any other advice would be greatly appreciated.

#!/usr/bin/perl

use strict;
use warnings;
use feature qw( say );

#Take input and output names from command line
my $in_qfn          = $ARGV[0];
my $out_qfn         = $ARGV[1];

my $transcripts_qfn = "/path/HumanTranscriptInfo";

my @transcripts;
{
   open(my $transcripts_fh, "<", $transcripts_qfn)
      or die("Can't open \"$transcripts_qfn\": $!\n");
   while (<$transcripts_fh>) {
      chomp;
      push @transcripts, [ split(/\t/, $_) ];
   }    
}

{
   open(my $in_fh, "<", $in_qfn)
      or die("Can't open \"$in_qfn\": $!\n");
   open(my $out_fh, ">", $out_qfn)
      or die("Can't create \"$out_qfn\": $!\n");
while (<$in_fh>) {
      chomp;

      #define what each column is to print later
      my ($ID, $strand, $chr, $start, $sequence, $quality, $numPositio
+ns, $mismatches) = split(/\t/, $_);

      #define an end position based on sequence length
      my $end = $start+length($sequence);
      
      #define useful info from reference file
      for my $transcript (@transcripts) {
         my $ref_chr    = $transcript->[0];
         my $ref_strand = $transcript->[6];
         my $ref_start  = $transcript->[3];
         my $ref_end    = $transcript->[4];
         my $info = $transcript->[8];
         
         #now compare values, and if there is a match for strand, chro
+mosome, and the start and stop within reference range range, print va
+lues
         if ($chr eq $ref_chr && $strand eq $ref_strand && $end >= $re
+f_start && $end <= $ref_end) {
            say $out_fh join("\t", $ID, $strand, $chr, $start, $end, $
+sequence, $numPositions, $mismatches, $info);
         }
         
         }
     }
}
}

[download]

In reply to How to make a hash to evaluate columns between large datasets by rambosauce

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.