I was tweaking a rather complicated data processing script of mine last week when I ran accross an output pipe I was using to massage the data: 
my $output_pipe = "|sort -n -t, -k3,4 | uniq > $tmpfile";
open(OUTFILE,$output_pipe) || die "could not open file $output_pipe fo
+r output: $!";

[download]
I said to myself: "self, why are you using two processes (sort and uniq) when one (sort with -u flag) would work? That's silly." So, even though it had nothing to do with what I was tweaking, I changed the output pipe ot be: 
my $output_pipe = "|sort -n -t, -k3,4 -u > $tmpfile";

[download]
and thought to myself "hah! got rid of one extra step."

Now, yesterday I noticed that the script was producing erroneous output. Since the input data often changes (depending on who produced it!) I have no formal spec for the input and I'm often tweaking the processing script to adapt it to new input formats (be liberal in what you accept). I thought that must have happened so I spent a good part of the day trying to track down what had changed. Since the data is several hundred MBs in size this process took a while.

Once I finally exhausted all possiblity that the input data was bad I turned to the script and finally I decided to test to see if "sort -u" was really equivalent to "sort | uniq"

my test data: 

$ cat test.dat
1,1
1,2
1,3
2,1
2,4
2,2
2,1
1,1

[download]

$ sort -n -t, -k1,2 test.dat | uniq
1,1
1,2
1,3
2,1
2,2
2,4

That's good.

[download]

$ sort -n -t, -k1,2 -u test.dat

1,1
2,1

Not what I expected.

[download]
Aha! With multiple keys, sort -u does not behave as I had assumed.

The moral of this little story: If it's not broken, then don't mess with it! (Oh, and don't assume either!)

Update: In light of the excellent comments and feedback here, I think I'll modify the moral of this story:

  1. Don't assume. Ever.
  2. Make sure you have unit tests.
  3. Run your unit tests. Frequently.
  4. Don't change something unless you have a good reason to.
  5. If you do change something, see rules 1,2,3,4 above

In reply to If it's not broken, don't fix it by RhetTbull

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.