Your code reads the entire INFILE2 for each line of INFILE.

I have re-written the code to help you get the logic right.

This code is fairly efficient (There is room for improvement, but it would increase complexity), and made more verbose and simplified.
See the discussion in Alternation vs. looping for multiple searches for improvement ideas.
Untested.

Assumes INFILE contents are smaller than INFILE2, AND small enough to fit into memory.

If this is not the case, you need to consider using a database.

#!/usr/bin/perl -w
use strict;
use warnings;

my %subst;
my $data = 'reference.txt'; 
open my $INFILE, '<', $data or die "cannot read $data: $!";

while(defined my $line = <$INFILE>)
{
   chomp($line);
    my ($id, $description) = split /\t/, $line, 2;
    $subst{$id} = $description;
}
close $INFILE;

my $data2 = 'readthisfile.txt'; 
open my $INFILE2, '<', $data2 or die "cannot read $data2: $!";

while(defined my $line2 = <$INFILE2>)
{
   ## chomp($line2); Dont chomp - leave newline in for printing
    my $changes = 0;
    for my $id (keys %subst){
        if ($line2 =~s/$id/$subst{$id}/g){
           # This line was changed
           $changes++;
        }
    }
    print $line2  if $changes; #Prints only changed lines
}
close $INFILE2;
[download]