here another approach:

repeating the tests according to their importance will create a metric at the end

use Test::More; sub weight (&$$) { my ($c_test, $name, $weight) = @_; my $res = subtest $name, $c_test; ok($res,$name) for 2.. $weight; } weight { is "a","a" } "PASSING" => 3; weight { is "a","X" } "FAILING" => 2; done_testing;

# Subtest: PASSING ok 1 1..1 ok 1 - PASSING ok 2 - PASSING ok 3 - PASSING # Subtest: FAILING not ok 1 # Failed test at /home/lanx/perl/pm/test_metric.pl line 16. # got: 'a' # expected: 'X' 1..1 # Looks like you failed 1 test of 1. not ok 4 - FAILING # Failed test 'FAILING' # at /home/lanx/perl/pm/test_metric.pl line 6. not ok 5 - FAILING # Failed test 'FAILING' # at /home/lanx/perl/pm/test_metric.pl line 8. 1..5 # Looks like you failed 2 tests of 5. <--- Metric

As you see, you are quite flexible with a bit of functional programming.

Another approach might be using Test::Builder to create your own semantics.

update
For instance you can use $Test->no_diag(0); to make the repeated fails less verbose.

update

or you can use my @tests = $Test->details; to get a detailed overview which tests passed and if they were todos.

# test_metric.pl:33: [ # { actual_ok => 1, name => "PASSING", ok => 1, reason => "", type = +> "" }, # { actual_ok => 1, name => "PASSING", ok => 1, reason => "", type = +> "todo" }, # { actual_ok => 1, name => "PASSING", ok => 1, reason => "", type = +> "todo" }, # { actual_ok => 0, name => "FAILING", ok => 0, reason => "", type = +> "" }, # { actual_ok => 0, name => "FAILING", ok => 1, reason => "", type = +> "todo" }, # ]

after putting the weight into the name, you can easily calculate your desired metric.

NB: ->details also works locally for subtests only.

Cheers Rolf
(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery


In reply to Re: Testing: shades of grey by LanX
in thread Testing: shades of grey by hv

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.