BDD strikes me as a variant of black box testing where the behavior of the "box" is defined using scenarios and language familiar to end users. The idea is to get end-users involved in the process of writing specs. Its chief advantage IMHO is expectation management: if the users are writing the tests, it is a lot harder for them to claim that the system doesn't do what they want it to do.

The problem I have with BBD is that its most vocal advocates seem to underestimate the difficulties in creating sound test suites. They sometimes argue as if talks with users are the only thing needed to focus the developer's attention and help them discover the "real API" (or at least the one that matters). I'm sure this true in part, but it isn't a substitute for good engineering and analysis. Users are a starting point for requirements, but not the ending point.

The "behaviors" most obvious to a user aren't always the most important for the long term stability of a system. If we limit testing to those scenarios we may have very unhappy customers 6 months down the road. Good test suites also need to handle things like boundary conditions that can only be found by analyzing the domains of inputs and calculating outputs. Story boarding simply isn't enough to get a full test suite. I find the claim that those non-user tests are low priority edge cases that no one cares about flimsy at best, especially since it is usually the non-obvious edge cases that end up as security vulnerabilities. Users don't always understand why they have to pay extra for tests they don't intuitively understand. But all that means is that part of our job as software engineers is to explain the value proposition behind the engineering side of development.

The situation is comparable to going to a doctor for a diagnosis. Sometimes patients have a good idea of what's wrong and a doctor would be very dumb to ignore the patients request that test X be carried out. Sometimes the patient is dead wrong, BUT the doctor knows that omitting that test will cause the patient to lose trust, so it gets done anyway. But which of us wants a doctor who only does the tests we ask? Or who makes us go through obscure tests whose value we don't understand?

That being said there are many CPAN modules for those interested in keeping tests close to user perceptions, API specs or in white box testing in general. Here is just a sampling:

• Test::LectroTest - framework for specification based testing. You can mix and match with Test::More, but see the Test::LectroTest::Compat module for details.
• Test::More::Behaviours - groups the assertions in test files into "behaviors" - really more of an organizational tool for your test suites.
• Test::FIT - supports FIT (Framework for Interactive Testing) for webpages. Although FIT is marketed as a competing testing methodology, FIT is one mechanism for converting test stories into test suites.
• Test::FITesque - another FIT framework
• Test::WWW::Declare - another tool meant to assist in converting stories and action sequences into tests.
• Class::Contract - supports OO design by contract.
• Test::ClassAPI - generates a test suite that verifies a contract
• Test::Role - verifies that all methods associated with a role have been defined.

All of these modules (CPAN links provided above) are part of distributions - to see the full list of modules in these packages, click on the distribution at the top of the CPAN link (the name with the dashes instead of ::).

Best, Beth

Update: clarification of role of users.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.