[Assertion failed] would be a lot more useful than [Test Failed]

I think this comment indicates that you don't have the same expectations of a test framework as I do. When I use a test framework I'm testing behaviour, such as the output of a function under certain circumstances. I'm not concerned with implementation im concerned with output and side effects. It really doesn't matter to me if Module.pm(257) expected to get a SCALAR ref and got something different, what matters to me is how that affected the output, or whether it prevented something from happening that should have happened.

As an example in DDS i have assertions in various places, as well as "insane case" handlers on the important branch points. However, knowing that one of these assertions failed doesnt help me nearly as much as knowing what input set lead to the assertion failing. Because of the structure of DDS I can end up executing those assertions on almost any input, so knowing that the assertion failed simply tells me that "for some unknown input something doesnt do the right thing". This doesnt help me nearly as much as knowing that "for a stated input something doesn't do the right thing". So when a test reports a failure I can easily identify the input code responsible, and then work towards the logic failure that caused it. Simply knowing that a logic failure occured doesnt help me.

When I download a module and it fails test the first thing i do is open up the test file to see what was happening then i look at the relevent code in the module and usually I can find a fix right away, with the definition of "fix" being that i can patch it so it passes test. I then continue on my way, satisifed that I know whats going on. On the other hand when I install a module and it fails I have to take considerable time to figure out why because i dont know if its me doing something wrong or if its the module doing something wrong, furthermore since I dont know exactly what it should be doing its much harder to fix it.

So I guess to me in simple terms regression tests are about why the code fails not so much what code failed. Or in other words its about identifying initial states that lead to failure, not about the exact internal cause of it.

---
$world=~s/war/peace/g

In reply to Re^9: Self Testing Modules by demerphq
in thread Self Testing Modules by Sheol

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.