in reply to Getting rid of duplicates
if you need a perl solution, try perl -e'my %list = (); $list{$_} = 1 while <>; print sort keys %list; ' < infile.txt