Re: Behavior Driven Development: suggested tools for perl?

BDD strikes me as a variant of black box testing where the behavior of the "box" is defined using scenarios and language familiar to end users. The idea is to get end-users involved in the process of writing specs. Its chief advantage IMHO is expectation management: if the users are writing the tests, it is a lot harder for them to claim that the system doesn't do what they want it to do.

The problem I have with BBD is that its most vocal advocates seem to underestimate the difficulties in creating sound test suites. They sometimes argue as if talks with users are the only thing needed to focus the developer's attention and help them discover the "real API" (or at least the one that matters). I'm sure this true in part, but it isn't a substitute for good engineering and analysis. Users are a starting point for requirements, but not the ending point.

The "behaviors" most obvious to a user aren't always the most important for the long term stability of a system. If we limit testing to those scenarios we may have very unhappy customers 6 months down the road. Good test suites also need to handle things like boundary conditions that can only be found by analyzing the domains of inputs and calculating outputs. Story boarding simply isn't enough to get a full test suite. I find the claim that those non-user tests are low priority edge cases that no one cares about flimsy at best, especially since it is usually the non-obvious edge cases that end up as security vulnerabilities. Users don't always understand why they have to pay extra for tests they don't intuitively understand. But all that means is that part of our job as software engineers is to explain the value proposition behind the engineering side of development.

The situation is comparable to going to a doctor for a diagnosis. Sometimes patients have a good idea of what's wrong and a doctor would be very dumb to ignore the patients request that test X be carried out. Sometimes the patient is dead wrong, BUT the doctor knows that omitting that test will cause the patient to lose trust, so it gets done anyway. But which of us wants a doctor who only does the tests we ask? Or who makes us go through obscure tests whose value we don't understand?

That being said there are many CPAN modules for those interested in keeping tests close to user perceptions, API specs or in white box testing in general. Here is just a sampling:

Test::LectroTest - framework for specification based testing. You can mix and match with Test::More, but see the Test::LectroTest::Compat module for details.
Test::More::Behaviours - groups the assertions in test files into "behaviors" - really more of an organizational tool for your test suites.
Test::FIT - supports FIT (Framework for Interactive Testing) for webpages. Although FIT is marketed as a competing testing methodology, FIT is one mechanism for converting test stories into test suites.
Test::FITesque - another FIT framework
Test::WWW::Declare - another tool meant to assist in converting stories and action sequences into tests.
Class::Contract - supports OO design by contract.
Test::ClassAPI - generates a test suite that verifies a contract
Test::Role - verifies that all methods associated with a role have been defined.

All of these modules (CPAN links provided above) are part of distributions - to see the full list of modules in these packages, click on the distribution at the top of the CPAN link (the name with the dashes instead of ::).

Best, Beth

Update: clarification of role of users.

Comment on Re: Behavior Driven Development: suggested tools for perl?

Replies are listed 'Best First'.
Re^2: Behavior Driven Development: suggested tools for perl? by Anonymous Monk on Apr 03, 2009 at 12:33 UTC
BDD isn't a replacement for decent testing at all. We use the scenarios to facilitate conversation, to ensure that the software we're writing is software that matters. Sometimes those scenarios can help the invaluable, irreplaceable QAs (but I've done them manually before, in the absence of a decent automation tool). We also identify stakeholders, rather than just users. If long-term stability, performance or security is important, there's probably a stakeholder who can define what that means, and how to tell if you've achieved it; they'd be the people we had that conversation with. Captchas are the example I use most frequently - users don't want them and won't care if they produce false positives. -- Liz.	[reply]
Re^3: Behavior Driven Development: suggested tools for perl? (Stakeholders) by ELISHEVA (Prior) on Apr 03, 2009 at 13:12 UTC
there's probably a stakeholder who can define what that means Indeed. As you know whenever something becomes a buzzword, it also becomes oversimplified. On the other hand, I question whether one is really doing BDD if one have broadened it to the general (non BDD specific) wisdom: listen to your stakeholders and agree upon measurable deliverables. The conversational tools of BDD are not necessarily the best communication method for all stakeholders. End users often respond well to stories because they experience their jobs as a flow. BDD can be an extremely effective tool for communication with this group of stakeholders. On the other hand, senior management tends to respond better to discussions that focus on cost-benefit analysis: the cost of implementing X versus the impact of X on market position, future sales, branding, and other strategic or operational goals. Stories (aside from case studies of market success or failure) tend to take the back stage. IT managers and the engineering staff who advise them are primarily interested in fact-based discussions of the impact of implementation X vs. Y on total cost of ownership (TOC), long term support needs and security related liabilities. Connecting the dots between the technical features of implementation X and the TOC of implementation X takes a lot more than getting stakeholders to articulate goals and scenarios. I find it far more important to do an assessment of organizational strengths and coping mechanisms. TOC does not exist in a vacuum - it is inseparable from organizational culture. I suppose you could call market impact, TOC or security risk level a "behavior", but if you do so, you've broadened the meaning of BDD so greatly that (a) behavior merely becomes a synonym for goals (b) BDD looses its distinctiveness as a methodology. Best, beth	[reply]