Re^2: What is a testing plan?

friedo wrote: The Extreme Programming people say that the tests are the specification.

I see that a lot, but it always concerns me because I don't think it really holds true. Consider the following tests:

can_ok $object, 'copy_for_xmission';
ok my $clone = $object->copy_for_xmission, '... and copying should be 
+successful';
isa_ok $clone, 'Some::Class';
ok !defined $clone->ssn,                   '... but the SSN should not
+ be copied';
[download]

It's very common to see that in tests and it's perfectly appropriate. Now in reading the test output, we can see that the SSN is not copied on clone, but we don't know why. There are plenty of reasons why we might not copy the SSN over, but the tests tend to reflect what is happening, not why it's happening. As a result, tests document behavior, not business rules.

The problem with this is institutional knowledge. In many companies I've worked for, documentation is almost an afterthought. Some other programmer looking at this might can see the behavior, but when getting asked to extend or alter the behavior, they can easily make mistakes without an understanding of the underlying business reasons for the code's behavior. Programmers often claim that you should simply be able to read the code (or tests) to know what's going on but that ignores that the larger the system, the larger the number of assumptions which might creep into the code without explanation.

Cheers,
Ovid

New address of my CGI Course.

Comment on Re^2: What is a testing plan? Download Code

Replies are listed 'Best First'.
Re^3: What is a testing plan? by xdg (Monsignor) on Mar 02, 2006 at 21:44 UTC
As a result, tests document behavior, not business rules. But that's not quite what friedo said. He wrote that "the tests are the specification". A specification just describes behavior -- the reasons why that is the desired behavior provide helpful context for interpreting behavior or generalizing it to new situations, but they aren't the specification itself. Assumptions creep into code because requirements are rarely well-specified. Therefore, they become open to interpretation -- which is why having that context and institutional knowledge documented becomes important. All TDD does is encourage developers to be explicit about how they are interpreting a requirement or specification, before they write the code. When people say tests or code are the documentation -- it's a limited form, but it has the advantage over other types of documentation that they are, by definition, describing what actually happens, rather than what was desired or intended, which which may or may not have been updated and which the program may or may not fulfill. But that's still just a documentation of the spec, not the requirements. What I find, personally, is that TDD helps brings focus. It's not a documentation technique; it's a task-management technique. There's a clearly defined concept of "done" and the code I write is limited to achieving "done" as directly as possible. The clarity of thinking through how to prove that something does what I expect also usually means that by the time I write actual code, I have a much better idea of what I need to write. That, too, saves time and improves the quality of what I write. -xdg Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.	[reply]

Replies are listed 'Best First'.

Re^3: What is a testing plan?
by xdg (Monsignor) on Mar 02, 2006 at 21:44 UTC

As a result, tests document behavior, not business rules.

But that's not quite what friedo said. He wrote that "the tests are the specification". A specification just describes behavior -- the reasons why that is the desired behavior provide helpful context for interpreting behavior or generalizing it to new situations, but they aren't the specification itself.

Assumptions creep into code because requirements are rarely well-specified. Therefore, they become open to interpretation -- which is why having that context and institutional knowledge documented becomes important. All TDD does is encourage developers to be explicit about how they are interpreting a requirement or specification, before they write the code.

When people say tests or code are the documentation -- it's a limited form, but it has the advantage over other types of documentation that they are, by definition, describing what actually happens, rather than what was desired or intended, which which may or may not have been updated and which the program may or may not fulfill. But that's still just a documentation of the spec, not the requirements.

What I find, personally, is that TDD helps brings focus. It's not a documentation technique; it's a task-management technique. There's a clearly defined concept of "done" and the code I write is limited to achieving "done" as directly as possible. The clarity of thinking through how to prove that something does what I expect also usually means that by the time I write actual code, I have a much better idea of what I need to write. That, too, saves time and improves the quality of what I write.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

[reply]