in reply to Getting rid of duplicates
We use the -lne command-line switches to cause Perl to read each line of input, strip off the line break, and then execute the following code on the result:perl -lne 'print unless $counts{0+$_}++' input.txt > output.txt
The code prints the current line if the count of times we have seen it so far is zero. We use the hash %counts to keep track of the counts. Note the 0+ inside of the hash index. It ensures that the input lines are interpreted as numbers so that, for example, "1" and "1.0" are considered to be the same for the sake of duplicate removal.print unless $counts{0+$_}++
Cheers,
Tom
Tom Moertel : Blog / Talks / CPAN / LectroTest / PXSL / Coffee / Movie Rating Decoder
|
|---|