Re^2: Testing at the right granularity

I agree, but I'm still dubious about the granularity problem I posed. As a matter of fact, I wasn't talking about coverage, corner cases and so on, but more about how a single "logical" test can result in a bunch of "atomic" tests.

To make an exageration, suppose I've some function result that I have to check against the string "ciao a tutti":

   my $result = function_under_test();
   my $expected = "ciao a tutti";
[download]

Now, I can take the direct path:

   is($result, $expected, "strings are equal");
[download]

But I can go much deeper:

   my @result = split //, $result;
   my @expected = split //, $expected;
   is(scalar(@result), scalar(@expected), "length match");
   for (my $i = 0; $i < @expected; ++$i) {
      is($result[$i], $expected[$i], "$i-th char match");
   }
[download]

I repeat: this is an exageration, I'm not saying that I'm doing my tests this way. Nonetheless, I sometimes feel that maybe a more compact test (like the first) would do equally good as the bloated one.

Flavio (perl -e "print(scalar(reverse('ti.xittelop@oivalf')))")

Don't fool yourself.

Comment on Re^2: Testing at the right granularity Select or Download Code

Replies are listed 'Best First'.
Re^3: Testing at the right granularity by revdiablo (Prior) on Apr 19, 2005 at 18:15 UTC
Nonetheless, I sometimes feel that maybe a more compact test (like the first) would do equally good as the bloated one. As long as they're testing the same things, yes. In my opinion, your 2nd snippet of code is by far inferior to the 1st. It is more verbose, and less clear. Keep in mind that writing tests still means following good coding practices. In this case, the simplest and most obvious solution is usually the best. If they were testing different things, though, then it's not so clear. I would lean towards testing as much as possible, as ambs said. Note that I am talking about this in terms of semantic tests, not in terms of number of "physical" test cases you can squeeze out. Update: dragonchild seems to also touch on this same idea in Re^3: Testing at the right granularity. If you're concerned about putting a lot of unrelated tests together, and thinking they may get drowned out in the noise, consider putting them in separate test files. This allows you to have your cake and eat it too. You get logical groupings, but also high granularity.	[reply]
Re^3: Testing at the right granularity by tlm (Prior) on Apr 20, 2005 at 02:32 UTC
I think the second set of tests misses the point of what a test suite should do, which is limited to making bugs visible. That's it! The analysis of the bug (which your second set of tests begins to do) should come after. All you want to do at this stage is to make it your program fail a test: the simplest the path there the better. (Of course, if you program does not fail despite your efforts, good for you, grab a beer.) It's like when you put a filled inner tube under water to quickly find the location air leaks (if any). That's all you'd expect from this test. You don't expect this test to tell you the diameter of the hole, or how it got there, or what it will take to fix it, or anything else. the lowliest monk	[reply]
Re^3: Testing at the right granularity by grinder (Bishop) on Apr 20, 2005 at 06:31 UTC
a single "logical" test can result in a bunch of "atomic" tests I agree with the other replies to this, but just to play devil's advocate... There can be a value in doing the things the second way, but it depends on factors that only you can judge. It comes down to a question of information gathering. If having a multitude of tests means you are able to pinpoint the exact point of failure in your code ("Hmm, the eighth character is uppercase, I guess that means someone optmi^broke the the tau_gamma() routine again") then the tests are worth doing. If only one of the atomic tests fail and the information is useful (helps you zoom in faster to the problem area), then do it that way. What you will usually find though, is that if the "logical" test fails, you will have a cascade of failing "atomic" tests as well, all pointing to the same problem. Here, no new information is being added to the picture. If this is the case then just do the logical test and be done with it. Remember to test for the boundary conditions. In Perl, I find this often means dealing with empty arrays `()`, references to empty arrays and lists (`[]` and `{}`, respectively), empty strings, undef and the values '0' and '0 but true' or 0e0. - another intruder with the mooring in the heart of the Perl	[reply]