repeating the tests according to their importance will create a metric at the end
use Test::More; sub weight (&$$) { my ($c_test, $name, $weight) = @_; my $res = subtest $name, $c_test; ok($res,$name) for 2.. $weight; } weight { is "a","a" } "PASSING" => 3; weight { is "a","X" } "FAILING" => 2; done_testing;
# Subtest: PASSING ok 1 1..1 ok 1 - PASSING ok 2 - PASSING ok 3 - PASSING # Subtest: FAILING not ok 1 # Failed test at /home/lanx/perl/pm/test_metric.pl line 16. # got: 'a' # expected: 'X' 1..1 # Looks like you failed 1 test of 1. not ok 4 - FAILING # Failed test 'FAILING' # at /home/lanx/perl/pm/test_metric.pl line 6. not ok 5 - FAILING # Failed test 'FAILING' # at /home/lanx/perl/pm/test_metric.pl line 8. 1..5 # Looks like you failed 2 tests of 5. <--- Metric
As you see, you are quite flexible with a bit of functional programming.
Another approach might be using Test::Builder to create your own semantics.
or you can use my @tests = $Test->details; to get a detailed overview which tests passed and if they were todos.
# test_metric.pl:33: [ # { actual_ok => 1, name => "PASSING", ok => 1, reason => "", type = +> "" }, # { actual_ok => 1, name => "PASSING", ok => 1, reason => "", type = +> "todo" }, # { actual_ok => 1, name => "PASSING", ok => 1, reason => "", type = +> "todo" }, # { actual_ok => 0, name => "FAILING", ok => 0, reason => "", type = +> "" }, # { actual_ok => 0, name => "FAILING", ok => 1, reason => "", type = +> "todo" }, # ]
after putting the weight into the name, you can easily calculate your desired metric.
NB: ->details also works locally for subtests only.
Cheers Rolf
(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery
In reply to Re: Testing: shades of grey
by LanX
in thread Testing: shades of grey
by hv
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |