For an arbitrary function foo, to test it, almost always, several tests are needed. The general idea being to supply in-range and out-of-range values for each of the inputs to foo. Is the simplest cases, each input has a defined valid range and you make sure that, at minimum, each input gets values above the maximum, below the minimum, just below the max, just above the min and 1 or more points distributed between max and min. Obviously, as the number of inputs increases, the combinatorics will quickly get impractical, so you have to come up with a reasonable and workable subset of tests.

In more complex cases, the (valid, in-range) value of one input can affect the valid range of another input. This is dependant on the intended behavior of foo

But, even with a well engineered test suite, it's still possible the tests won't protect you from changes to even a very thoroughly tested function. It's still possible to miss corners (and other nexi) of the n-dimensional hyper-box.

Even assuming your existing test suite is well engineered, you are very likely to find that it won't protect you from code changes, however reasonable they are.

When looking at the code and planning a change, think about what might go wrong, then create more tests to cover those. Then after a reasonable number of changes have been made, hold a code review. Even if you have only yourself to do the review. Sometimes, after a few days and working on (or looking at) other code, you can go back and be able to see problems you couldn't when you made the changes.

In reply to Re: Testing my tests by RonW
in thread Testing my tests (mutation testing) by szabgab

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.