Here is a way that will work, without using modules.
This design is limited to comparing 31 files (assuming a 32 bit machine). Yes - I could extend it to 32 files at the expense of even more obfu.

my @file1 = qw{1 2 3 4  }; 
my @file2 = qw{  2 3 4 5}; 
my %seen;
my @fa =([1<<1,\@file1,'File-1 only'],[1<<2,\@file2,'File-2 only'],
      [1<<1|1<<2,[],'Both have']);

for (@fa){
  my ($v,$f)=@$_;
  $seen{$_} |= $v for @$f
};
 ## print Dumper \%seen;

for (@fa){
  my ($v,$f,$n)=@$_;
  $v==$seen{$_} and print qq[$n :$_\n]  
      for sort keys %seen
}
[download]

Prints:

File-1 only :1
File-2 only :5
Both have :2
Both have :3
Both have :4
[download]

Update: Updated code to also print "Both" lines.