Yeah - I saw that TBD stuff. But others haven't and the purpose of posting a node on Perl Monks is to create a discussion from which we can all, not just the OP, learn.
Under ideal circumstances I prefer to have different people do writing and testing - it also helps a lot in identifying documentation errors and fuzzy specs. Often the person who does the coding is so close to the problem that they are unaware of their implicit assumptions. But small teams don't always have that luxury. Given that much important software innovation comes from under-resourced start-ups, skunk teams within corporations, and open-source projects, I think it is important to develop testing philosophies that work for teams both large and small.
As for tautology testing I have mixed feelings. As gwadej pointed out (and LanX echoed) there are other reasons for testing (regression, crash testing) and they are very important. If your software is going to have a life cycle with new features and patches then regression testing is reason enough to pay the cost of test development.
As I ponder the discussion so far, I'm beginning to realize that many things that seem like tautologies are not actually tautologies. It all depends on what you are using the test for and why. As long as we are very clear on what the test can and cannot verify, the test may still be valuable.
For example, as moritz discusses, testing that "true is, well, true" is still a valuable test if you are testing a compiler because there are an unbelievable number of ways to screw up a compiler. The test seems like a tautology only when we discount the amount of processing involved for a compiler to decide that True is true. gwadej makes a similar point when he discusses debugging things that "could not possibly be wrong".
Or take an even more extreme example, also given by moritz: testing sub s. If the purpose of the test is to verify that something is correctly split then using split to test a sub that calls split is a very bad idea. However, the purpose of such a test may be something very different.
Suppose you have API documentation that says that parameters should be delivered in a particular order and you want to verify that sub s indeed expects those parameters. A true Ttest (to borrow gwadej's term) would pass when even the code is wrong. Parameters in the wrong order will not pass if they are wrong! Hence, for purposes of parameter order testing, even moritz's sub s/split example isn't a tautology.
In conclusion, I think at least three questions need to be answered to decide if Ttesting is overtesting:
As I think about what as been said so far on this thread, I am honestly surprised at how many non-technical factors and trade-offs seem to be creeping into the decision process of deciding what and how to test. Your post does a good job of stressing that point. So far we have:
I wonder what others will appear.
Best, beth
In reply to Re^2: How does one avoid tautologies in testing?
by ELISHEVA
in thread How does one avoid tautologies in testing?
by ELISHEVA
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |