I hate to have to answer my own question, but I figured it out.

The default Makefile.PL that is supplied uses inc::Module::Install which is just a thin wrapper around Module::Install. There was some promising-looking but highly misleading code in Module::Install::Makefile. When I got past that, it turns out that I seem to be getting the default behaviour from ExtUtils::MakeMaker. Which is to run everything of the form t/*.t, and to also execute a file called test.pl if one exists.

My solution was therefore to create a file called test.pl that looks like this:

# Run all of the tests below the lib directory.
exec(qw(prove --lib lib --recurse lib));
[download]

Ugly and hackish, but it works.