What I need to do is be able to read the data based on the SECOND_NUMBER.

Sounds like you want an array of records for each value SECOND_NUMBER can take.

my $header = <>;
chomp($header);
my @field_names = split  /\t/, $header;

my %grouped_by_num2;
while (<>) {
    chomp;
    my %rec; @rec{ @field_names } = split /\t/;
    push @{ $grouped_by_num2{$rec{SECOND_NUMBER}} }, \%rec;
}

use Data::Dumper;
print(Dumper(\%grouped_by_num2));
[download]

So, in the code you suggest, ID won't be used at all? I don't need to use ID after all, I just want to store the pairs SECOND_NUMBER<->FIRST_NUMBER, so It suits me better...
However, how would you use the code you provide in order to get the pair 9<->6 in line 12? If I gave you $wanted_second_number=9, how would you print $first_number=6?

So, in the code you suggest, ID won't be used at all

It's there, just not used as the index.

However, how would you use the code you provide

The output of Data::Dumper illustrates that quite well.

If I gave you $wanted_second_number=9, how would you print $first_number=6?

No really, look at the structure yourself before continuing.

my @matching_recs = $grouped_by_num2{$wanted_second_number};

for my $rec (@matching_recs) {
    my $first_number = $rec->{FIRST_NUMBER};
    print("$first_numner\n");  # 5, 6
}
[download]

Your comments suggest you want to be able to look up by either first or second number. For this, if you want to use hashes, you can build two hashes to support lookup by either number.

Alternatively, if you don't have too many records, you could simply populate an array with all the records and select from this array as necessary.

Both approaches are demonstrated in the following example:

use strict;
use warnings;
use Data::Dumper;

my %hash;
my @records;

while (<DATA>) {
    chomp;
    my ($id, $first, $second) = split /\s+/;
    my $record = [ $id, $first, $second ];

    # Build hashes indexed by first and second numbers
    push(@{$hash{first}{$first}}, $record);
    push(@{$hash{second}{$second}}, $record);

    # Build array of records
    push(@records, $record);
}

print "\%hash:\n";
print Dumper(\%hash);
print "\n\n";

print "\@pairs:\n";
print Dumper(\@records);
print "\n\n";

print "records with first number 6, from hash:\n";
foreach my $record (@{$hash{first}{6}}) {
    print "\t", join(',',@$record), "\n";
}

print "records with first number 6, from array:\n";
foreach my $record (grep { $_->[1] == 6 } @records) {
    print "\t", join(',',@$record), "\n";
}

__DATA__
1    1               1
2    1               2
3    2               3
4    2               4
5    3               5
6    3               6
7    4               7
8    4               6
9    5               8
10    5               9
11    6              10
12    6               9
[download]

The hash will provide faster lookup if you have many records but the difference will be small if you have only a few records, as in your example.

You said ID isn't unique, and you said ID is unique. Which one is it? It's a crucial detail in determining the answer.

(Answered over here as I was typing)

Yes, my mistake...
The format is as you see it, that is ID is UNIQUE, while FIRST_NUMBER and SECOND_NUMBER are not unique...

If you are to filter by either first number, second or id I'd suggest to use

$hoh->{$id} = { first_number  => $first_num,
                second_number => $second_num};
[download]

and then to filter simply use grep

# Retrieve all IDS were first number is 2
my @ids = grep { $hoh->{$_}->{first_number} == 2 } keys $hoh;
[download]