#!/usr/bin/perl

use warnings; 
use strict;

open(KEY, "<hashKey.txt") 
    or die "error reading key list";
open(REG, "<testReg.txt") 
    or die "error reading file";

my %Chr;
while (my $key = <KEY>) {
    chomp $key;
    $Chr{$key} = undef;
}

my %R;
while (my $reg = <REG>) {
    chomp $reg;
    my @reg_split = split("\t", $reg);
    push @{$R{$reg_split[0]}}, $reg;
}

foreach my $key (sort keys %R) {
  next unless exists $Chr{$key};
  for my $out (@{$R{$key}}){ 
    print "$out\n";
  }
  print q{-} x 20, qq{\n};

}

close(KEY);
close(REG);
[download]

chr1    100    159    0
chr1    200    260    0
chr1    500    750    0
--------------------
chr11    679    687    0
--------------------
chr22    100    200    0
chr22    300    400    0
--------------------
chr3    450    700    0
--------------------
chr4    100    300    0
--------------------
chr7    350    600    0
--------------------
chr9    100    125    0
--------------------
[download]

The second while loop creates a hash of arrays (%R) from your input file. The key is the first field (chromosome) and the value is an array of records. That's what the push is doing.

Finaly we print the records for each chromosome if it exists in the lookup table. In your case you want to print to a file rather than STDOUT as we do here.

As an aside, you could rewrite the first while loop with map.

Hope that helps.

Update
Reading your question again I see

hashKey.txt gives a list of all the possible chromosome values there could be in a given input file.

And for the rest, as wfsp already said: push the data onto the anonymous array which comprise the values of the hash (autovivified, so don't worry about the anonymous array not existing).

while (<REG>) {
    chomp;
    my($key, $data) = split "\t", $_, 2;
    push @{$R{$key}}, $data;
}
[download]

foreach my $key (keys %R) {
    open my $fh, '>', "$key.out" or die "Cannot open file $key.out: $!
+";
    foreach my $row (@{$R{$key}}) {
        print $fh "$key\t$row\n";
    }
}
[download]

You are over-thinking this. You don't even need the file: hashKey.txt. The file testReg.txt is I think the 15GB monster file. If this file is not already sorted, use the system command line sort to do that. The command line sort can sort things way bigger than the size of memory.

Now all of the lines that have the same chromosome will be grouped together in the file. We just read the file and every time we switch to a new chromosome, we start a new file.

#!/usr/bin/perl -w
use strict;

my $curr_chrom = "";

while (<DATA>)
{
   my ($chrom) = split;  # $chrom is the first column
                         # parens on the left side are needed
                         # for list context
   if ($chrom ne $curr_chrom)
   {
       $curr_chrom = $chrom;
       open (OUT, '>', "$curr_chrom.out") 
          or die "unable to write $curr_chrom.out $!\n";
   }
   print OUT;
}
close OUT;


__DATA__
chr1    100 159 0
chr1    200 260 0
chr1    500 750 0
chr3    450 700 0
chr4    100 300 0
chr7    350 600 0
chr9    100 125 0
chr11   679 687 0
chr22   100 200 0
chr22   300 400 0
[download]

Update: From the wording of the post, I don't think that you are interested in a subset of the chromosomes in the input file, but if you were, then here's how. Make a hash table with keys being the chromosomes that you want. In the above program, when the chromosome changes (the if statement), test if the chromosome is on the "approved" list (name exists in the hash table) or not. If it does exist, then open OUT to that name like above, if it does not, then open OUT to "/dev/null". /dev/null is a special device that discards all stuff written to it (it is the "bit bucket"). That way you always execute the print OUT; statement. Sometimes it goes somewhere useful and sometimes into the black hole of bits.

To make the hash, your code:

while (<KEY>) {

    chomp;
    @key_split = split("\n");
    $Chr{"$key_split[0]"} = $key_split[0];
}
## better written as: ##
while (<KEY>) {
    my ($chrom) = split;
    $Chr{$chrom}=1;
}
[download]

Marshall,

I suppose I was over-thinking it. Your method looks to read in constant time, and for a large input file that I'm working with I think that may be beneficial.

The only reason I included the key list is because I thought that would be the easiest way to separate the input data into separate files. However, the input data is already well-sorted so your method should work.

One more question in general: with my code and the suggestions everyone has given, there is still an error message (despite the fact that the output is correct). Is there any way to get rid of this error message or is it just something I am going to have to deal with? The message refers to uninitialized value errors, and I was trying to fix this before. However, I suppose if the output is still correct that is the only thing that matters.

Yes, once sorted, the algorithm just reads the file once in a linear fashion. So this should be great for your humongous file.

The warning message should give you a line number in the code and often you also get the line number of the input file. One common way to get an uninitialized value is when there is a blank line in the file - this causes the split to fail (no results). An extra carriage return is easy to get missed since they are "invisible". I often put: next if /^\s*$/; which will go to the next input line if the current line contains nothing but white spaces.

I think that I already mentioned that normally you probably should be splitting on the regex /\s+/ which is the default. white space (\s) includes all of the following: the space of course,\n\r\f\t any contiguous sequence of those gets removed. Splitting on just tab characters (\t) can cause problems if there are sometimes extra space characters in there that you cannot see with the editor.

I think you are on the right track - keep at it!