Writing tests is a pain in the backside. If you are to write a good test suite you obviously need to test the algorithms used in your methods - that is, make sure that it returns the right results. But you also need to test that your code fails correctly. Given that you are a good programmer, you will have documented all your failure modes. I find that my code always has far more ways of failing than of succeeding, and that most of those are due to the programmer supplying bogus arguments to my methods. I might document a simple method like so (taken from a user/group management application: 
=item addgroup

Takes one parameter, which may optionally be named 'group'.
We detect whether it is a named parameter or not by checking
for the presence of 'group' followed by anything else in the
arguments.  This should be a valid groupname (ie consists
solely of word characters) which does not exist.  If that's
OK, an empty group is created.  All errors are fatal.

[download]
So let's see what are the failure conditions here:

There is one way of succeeding - if none of the above apply, an empty group is created. And if any of the above apply, we die().

I want to do as much as I can to automate the writing of my tests. Clearly, I can't automate writing tests for success, but I feel that I should be able to at least go part of the way towards automating tests for failure.

I would like to embed those tests in the POD in such a way that I can use a tool to both spew out a skeleton for the tests to which I can later add tests for success and for any other failure conditions which can't be easily specified (eg, in this case the test for "user specified a group that already exists"); and also the same data that was used to generate those tests will also generate human-readable documentation. I'm prepared to accept human-readable docs that are somewhat stilted.

There is already Test::Inline on the CPAN, but that doesn't go far enough - it lets me write my tests inline with the POD, but does not attempt to generate them automagically.

I'd like feedback on whether people think this is a good idea, and on the sort of syntax I should use.

In reply to Automatic generation of tests by DrHyde

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.