Your immediate problem is that you read all of file_compare in the first iteration of the outer loop while handling the first clients line, but don't reset to the start of the file_compare files for the next iteration.

However, that is a really bad way to do it. You end up reading the file_compare file right through for every line of the clients file which likely to be very slow! Much better is to build a list of clients on a pass through the client list file, then use that list in a separate loop for processing the log file. Consider:

use strict;
use warnings;

my $clientList = <<DATA;
fred
bob
DATA

my $backupData = <<DATA;
joe,2,3,wilma
fred,1,3,wilma
bob,2,3,freda
freda,7,2,fred
DATA

my $outputData;

open (my $clientsFile, '<', \$clientList) || die "Can't open clients: 
+$!\n";
my @clients = <$clientsFile>;
close $clientsFile;
chomp @clients;

my $match = '\b(?:' . (join '|', @clients) . ')\b';

open (file_compare, '<', \$backupData)
    || die "Can't open file_compare: $!\n";
open (CSV_OUT, '>', \$outputData) || die "Unable to open lastbackup.cs
+v: $!";
print CSV_OUT "SERVER,CLIENT,LAST SUCCESSFUL FULL,LAST SUCCESSFUL INCR
+,LAST ATTEMPT\n";

while (<file_compare>) {
    if (m/$match/) {
        print CSV_OUT "$_";
    }
}

close CSV_OUT;
close file_compare;
print $outputData;
[download]

Prints:

SERVER,CLIENT,LAST SUCCESSFUL FULL,LAST SUCCESSFUL INCR,LAST ATTEMPT
fred,1,3,wilma
bob,2,3,freda
freda,7,2,fred
[download]