G'day sedusedan,

"I'm wondering if someone can point me to some simpler examples of testing a Test::Builder-based test function."

You ask for "simpler examples" but you don't indicate simpler than what. The EXAMPLES section of the Test::Builder documentation says: "CPAN can provide the best examples. Test::Simple, Test::More, Test::Exception and Test::Differences all use Test::Builder.". Are you looking for simpler examples than these? There are links to dozens of tests in the Test::Simple MANIFEST. Have you looked at those?

"I glanced at Test::More's tests once a while back and decided not to do it yet because they looked pretty hairy."

I'm not sure whether you're referring to the tests documented in Test::More or the tests written in t/More.t (or, possibly, something else). Perhaps start with Test::Tutorial. You can always ask here if there's something specific you're having problems with.

I'm referring to Test-Simple's tests (yes, like the file you mentioned, t/More.t). Basically I have this test_foo() function that uses Test::More and does several calls to ok(), is(), is_deeply() and subtest() inside it. I want to test that test_foo() works. Specifically, aside from making sure that test_foo() can pass a test, I need to test that test_foo() can also fail a test. Something like:

#test_test_foo.t
#!perl
use Test::More;
ok( test_foo(a=>1), 'test_foo() should succeed if given a=1'); #1
ok(!test_foo(a=>2), 'test_foo() should fail if given a=2'); #2
done_testing;

That's the tricky part for me, since doing #2 will fail test_test_foo.t.

BTW, after looking at t/More.t, I also don't see that it tests failures like #2.

Untested: What about

ok(! eval { test_foo(a=>2) ; 1}, 'test_foo() should fail for a=2');
is($@, 'invalid argument', 'expected error');
[download]

لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ

Test::More comes with Test::Builder::Tester for testing testing modules. It's horrible. Just horrible.

Use Test::Tester instead. Here's an example of using Test::Tester to test the is_deeply function from Test::More, and make sure there isn't some insane bug in is_deeply that nobody's ever noticed...

use strict;
use warnings;
use Test::Tester;
use Test::More 0.96;

subtest "two identical structures" => sub {
    my (undef, $result1) = run_tests sub { is_deeply( [1,2,3], [1..3] 
+) };

    ok(
        $result1->{ok},
        'is_deeply passes',
    );

    is(
        $result1->{diag},
        '',
        'no unnecessary diagnostics printed',
    );

    done_testing;
};

subtest "two different structures" => sub {
    my (undef, $result2) = run_tests sub { is_deeply( [1,3], [1..3] ) 
+};
    
    ok(
        !$result2->{ok},
        'is_deeply fails'
    );
    
    like(
        $result2->{diag},
        qr{Structures begin differing},
        'expected diagnostics ok'
    );
    
    done_testing;
};

done_testing;
[download]

TL;DR: run_tests runs some tests in a coderef, captures the TAP output, and turns it into a list of hashrefs of test results. You can then use plain old Test::More functions to test those test results.

Thanks Toby, Test::Tester is exactly what I want. I'm gonna take your word on Test::Builder::Tester being horrible, besides it's a bit too low-level for my taste.

I ended up not using Test::Tester because it has problems with subtests (submitted as wishlist to RT), so I currently run the test in a subprocess and parse the TAP output manually.

I'm confused (i don't understand the question), are you really asking how to test your testing-library-functions?

Yup, sorry for not being clearer. My module in question is Gen::Test::Rinci::FuncResult. The module actually *generates* test functions, e.g. test_foo() or test_bar(). The test functions basically do a Test::More::subtest() with some assertions like ok(), is(), is_deeply() inside it.

Are you saying that I need to mock Test::More's ok()? I also use subtest(), is_deeply(), is(), like() BTW, should I mock those too? Why? I'd rather execute the test in a subprocess and parse the TAP instead.

You could do that, but beware that the TAP standard is slowly evolving. You need to make sure that you can cope when new features are added to it. You may also need to cope with code that doesn't actually use Test::More and just outputs something vaguely TAP-ish. Lots of old code's test suites contain things like this:

print 'not ' unless($foo == 42);
print 'ok '.$test++."\n";
[download]

instead of

ok($foo == 42);
[download]

Write a test script that includes successes and failures but don't name it as a test script in your test suite. Add to your test suite a test that runs the above test script and verifies that it outputs "ok 1" and "not ok 2" or whatever. Simple.