The following one-liner does what you want and has the advantages of handling the multiple representations available for numbers and preserving the order of the input lines. I expect that both are important to you because you didn't just use sort -nu to solve your problem: 
perl -lne 'print unless $counts{0+$_}++' input.txt > output.txt

[download]
We use the -lne command-line switches to cause Perl to read each line of input, strip off the line break, and then execute the following code on the result: 
    print unless $counts{0+$_}++

[download]
The code prints the current line if the count of times we have seen it so far is zero. We use the hash %counts to keep track of the counts. Note the 0+ inside of the hash index. It ensures that the input lines are interpreted as numbers so that, for example, "1" and "1.0" are considered to be the same for the sake of duplicate removal.

Cheers,
Tom

Tom Moertel : Blog / Talks / CPAN / LectroTest / PXSL / Coffee / Movie Rating Decoder

In reply to Re: Getting rid of duplicates by tmoertel
in thread Getting rid of duplicates by New Novice

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.