This is a curiosity question. A couple of times here at the Monestary, I've come into a thread and worked up a test framework for the various solutions posted. Here are examples:

• Re: creating a variable in an if statement
• Re: Golf/Perlish solution to this problem?

In a single script, I send multiple tests to each of multiple solutions. Pseudocode:

foreach my $solution ( ... ) {
    foreach my $test ( ... ) {
        ok( ... );
    }
}
[download]

The result is that I get multiple lines of output for each solution when the test script is run. The problem is that I'm really only interested in the solution as a whole. Did it pass all tests or not? The solutions are not otherwise related to each other. I'd like to get a report like Test::Harness would give for separate test scripts. I'd like to have something notionally like this:

foreach my $solution ( ... ) {
    harness "tests for $solution" {
        foreach my $test ( ... ) {
            ok( ... );
        }
    }
}
[download]

Then instead of:

ok 1 - kyle test 1
ok 2 - kyle test 2
ok 3 - elyk test 1
not ok 4 - elyk test 2
# four
# lines
# of
# stuff
[download]

...I could get something like this:

kyle...ok
elyk...dubious
[download]

It would be nice if there were a flag I could set to tell it whether to summarize results or show individual tests. I can think of two ways to get something like what I want, but they're both more pain than I'm willing to endure.

One is to write separate files for each solution. I could put each solution in a heredoc, write them out to a bunch of test files and have Test::Harness actually execute them all individually.

The other is to not execute individual tests as tests. Instead of ok() everywhere, just check the test results manually and call a single pass() or fail() for each solution. If I did this, I'd probably end up writing my_own_ok(), etc. I'd do this so that it would be easy to run it as a full test (without the harness) if I'm interested in the details of one solution.

Is there a better way to do what I'm looking for?