in reply to Removing duplicate lines from files (was 'Files')
perl -ne 'print if (++$a{$_} == 1);' <oldfilename >newfilename [download]
--- -DA [download]