Ok, try this below. You need an array to describe file2 so that you can loop over it for each entry in file1. use strict; and use warnings; as a huge help. I would only use all CAPS for bareword file handles. I changed the names accordingly.

#!usr/bin/perl
use warnings;
use strict;
use Data::Dumper;

my $file1 =
'A  1_1
A  1_2
B  1_3
C  1_4
';

my $file2 = 
'A  2_1
B  2_2
';

open FILE1, "<" , \$file1 or die "unable file1 $!";
open FILE2, "<",  \$file2 or die "unable file2 $!";

my @file2;        #need to compare each line in file1 against all
while (<FILE2>)   #lines in file2, so create an array of file 2
{
   chomp;
   push @file2, $_;
}


while (<FILE1>) 
{
    chomp;
    my ($file1letter, $file1number) = split (' ', $_);
    
    foreach my $line2 (@file2)
    {
       my ($file2letter, $file2number) = split (' ', $line2);
       
       if ($file1letter eq $file2letter)
       {
          print "$file1letter from FILE1 with number $file1number and 
+$file2letter from FILE2 with number $file2number match\n";
    
       }
       else
       {
          print "$file1letter from FILE1 with number $file1number and 
+$file2letter from FILE2 with number $file2number DO NOT match\n";
     
       }
    }
}

__END__

A from FILE1 with number 1_1 and A from FILE2 with number 2_1 match
A from FILE1 with number 1_1 and B from FILE2 with number 2_2 DO NOT m
+atch
A from FILE1 with number 1_2 and A from FILE2 with number 2_1 match
A from FILE1 with number 1_2 and B from FILE2 with number 2_2 DO NOT m
+atch
B from FILE1 with number 1_3 and A from FILE2 with number 2_1 DO NOT m
+atch
B from FILE1 with number 1_3 and B from FILE2 with number 2_2 match
C from FILE1 with number 1_4 and A from FILE2 with number 2_1 DO NOT m
+atch
C from FILE1 with number 1_4 and B from FILE2 with number 2_2 DO NOT m
+atch
[download]

update: I didn't see the code from pryrt before I hit the send button. The code above reads FILE2 into a memory array. Opening and reading a file is an "expensive" operation both CPU and time wise. Unless FILE2 is humongous, reading it once from the disk is the best approach.