...by concentrating on an intelligently selected subset of all possible tests, you can increase the effectiveness of the testing whilst reducing the test time. The shorter the test time, the more frequently it is likely to be run.

To an extent I agree, but not totally. For most of the modules I've written the past year or so, I've found that I actually need tests for 3 contexts:

1. Does it seem to work in the user's environment?
This is the test that you want typically run in a CPAN(PLUS) environment when a user installs a module. It should be relatively throrough, but shouldn't take too long.

2. Does it survive stress testing?
For most linear programs not really an issue, but if you're working on threaded programs like I have the past year, you know something may appear to work under "normal" situations, but will break down when being hit with everything you got. ;-(

3. Do all possible combinations of parameters work?
This is the test that you have internally to make sure that whatever combinations of parameters that can exist, that something valid happens. This is especially important with combinations of command line parameters. This can run as long as you need: for one project I'm working on, this typically runs for 45 minutes: not something you would like your average user to go through.

Most of my modules on CPAN have at least tests of category 1. Some of them have in category 2 as well. Category 3 tests I only run internally or by a user who is experiencing specific problems, not during a standard test/install sequence.

Liz

In reply to Re: Re: TDD: a test drive and a confession by liz
in thread TDD: a test drive and a confession by dragonchild

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.