It really depends on what you want to do in the end. Do you want to create a new file with all the duplicate entries removed? Do you want to keep one instance of each unique entry? Or do you simply need a report about the entries?

From what you told us, I'd say there's no need for reading up the entire file in memory, something like the following should do.

#!/usr/bin/perl
use strict;
use warnings;

my %seen;
my $file = "/path/to/your/file";

open(MYFILE, "<", $file) or die "Could not open '$file' for reading: $
+!";
while (<MYFILE>) {
    $seen{$_}++;
}
close MYFILE;

# Now every unique entry has a value of 1 in the hash %seen
print "Unique entries:\n";
print "$_\n" for (grep { $seen{$_} == 1 } keys %seen);
[download]