Finally you can also use TAP::Parser to scan the output.

So if I were you, I'd use the demonstrated abstraction in combination with Test::Builder to write your own appropriate test-semantic and put the "weight" into the comments.

I would than parse the verbose output and calculate the desired "metric" from the weights, with TAP::Parser.

I wouldn't mess with the classic test result output, cause it's a different "all or nothing" metric apart from the "better" metric.

So better put non-crucial tests into TODO blocks, because they are not kill criteria.

Regarding testing things with various level of "ok-ness", I'd write a test which tests them from best to worst and bails out as soon as it passes -i.e. reporting OK-and logging the weight in the comments.

Like if-elsif-... chains or in case of simple strings hash look-ups with the weight as values.

A routine like

isinchain( got, [ [exp1,weight1,name1], etc... ] , name) could do the abstraction

That's all very abstract because you're description was very abstract too.

Hope this helps. :)

Cheers Rolf
(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery

In reply to Re^2: Testing: shades of grey by LanX
in thread Testing: shades of grey by hv

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.