Finally you can also use TAP::Parser to scan the output.

So if I were you, I'd use the demonstrated abstraction in combination with Test::Builder to write your own appropriate test-semantic and put the "weight" into the comments.

I would than parse the verbose output and calculate the desired "metric" from the weights, with TAP::Parser.

I wouldn't mess with the classic test result output, cause it's a different "all or nothing" metric apart from the "better" metric.

So better put non-crucial tests into TODO blocks, because they are not kill criteria.

Regarding testing things with various level of "ok-ness", I'd write a test which tests them from best to worst and bails out as soon as it passes -i.e. reporting OK-and logging the weight in the comments.

Like if-elsif-... chains or in case of simple strings hash look-ups with the weight as values.

A routine like

isinchain( got, [ [exp1,weight1,name1], etc... ] , name) could do the abstraction

That's all very abstract because you're description was very abstract too.

Hope this helps. :)