in reply to Re: Automatic generation of tests
in thread Automatic generation of tests

The Devil's Advocate is good - the term comes from a church official appointed to research and present arguments against a proposed sainthood to ensure that the case *for* that sainthood is as watertight as possible :-)

I also posted this to the London Perl Mongers list, and from the feedback there, it seems that I wasn't quite as clear as I should have been. I see this as being a way to auto-generate the repetitive simple tests that I hate writing - exactly the ones that are needed for the example I gave. Any more complex method still needs all those tests to make sure that it is getting at least part of its argument sanity-checking correct. I do not propose that any such mechanism should completely obviate the need to write some tests by hand.

Even if I use Params::Validate or Class::Contract I'd still need to write those sorts of tests, just to check that I've used those modules correctly.

Your point about it maybe making the module harder to read is a good one, although one which can - I hope - be solved by choosing the right syntax.

Replies are listed 'Best First'.
Re^3: Automatic generation of tests
by adrianh (Chancellor) on Feb 23, 2004 at 15:33 UTC

    Can you give an example of the kind of things you would want to generate? I'm not getting a clear idea of what you're expecting.

    My first reaction is that the need to generate repetitive tests would be a code smell indicating that some abstraction in the code is being missed - but that may be because I'm missing your point :-)

[reply]
      For the example I gave: 
      =item addgroup

Takes one parameter, which may optionally be named 'group'.
We detect whether it is a named parameter or not by checking
for the presence of 'group' followed by anything else in the
arguments.  This should be a valid groupname (ie consists
solely of word characters) which does not exist.  If that's
OK, an empty group is created.  All errors are fatal.

      [download]
      I'd want to somehow encode that so that it could spew out the tests to make sure it dies if the number of arguments is wrong or if the one significant argument matches /\W/. This would be several tests. I would then add tests manually to make sure it dies if the group already exists, and to make sure it creates a group correctly.
[reply]
[d/l]
        I'd want to somehow encode that so that it could spew out the tests to make sure it dies if the number of arguments is wrong or if the one significant argument matches /\W/. This would be several tests.

        So you're proposing a domain specific mini-language that you can use to specify the arguments that your subroutine takes, and then use that to generate the code for the tests. You then edit the generated tests too add stuff that's not handled by the mini-language. Yes?

        I'm suspicious of generated code that needs editing. If you have to update the functionality at a later date then you need to regenerate the code, and then reintegrate the edits - which can be a painful process.

        Wouldn't it be simpler to implement your mini-language as a Test::Builder based Test:: module and skip the whole code-generation side?

        Hopefully this makes some vague sort of sense :-)

[reply]

In Section Meditations