This looks very interesting. Writing tests that are valid for different subroutine signatures will be a challenge, though. Your example above creates a test where each subroutine is excercized with an empy argument list. Generating the input with non-empty argument lists is non-trivial. What happens when you have five subroutines that expect hashrefs and one that expects a scalar? I've pondered this problem a bit earlier, and perhaps it is possible to add more code to parse subroutine signatures and feeding the results to some sort of test factory. This will at least reduce overhead in writing new test types.

Your title includes "Edge Testing". By concentrating on edge cases, i assume that you attempt to improve code coverage. Perhaps you could piggyback on Devel::Cover to get statistics on your code? Devel::Cover presents its data in a structured way, which is (normally) used to generate the pretty HTML-presentations. Maybe it is possible to analyse test input, and what branches are excercised, and then to generate more input data to excercise the uncovered branches? This does not sound trivial, but then again, your solution isn't either ;)

Lastly, I think your code parsing is a bit fragile. Regular expressions are wonderful, but Perl syntax is notoriously flexible. Therefore, reuse of your work depends on a code standard. This is fine, of course, as powerful tools will put restrictions on what you are allowed to do (viz static analysis tools). Still, Perl hackers are notoriously TIMTOWTDI. Maybe PPI can help with the parsing?

Your code is very interesting, and I wish you good luck in your project. Please keep us posted as you progress :)

pernod
--
Mischief. Mayhem. Soap.

In reply to Re: Automated Edge Testing by pernod
in thread Automated Edge Testing by thekestrel

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.