Take a look at Test2::Plugin::BailOnFail. You can probably copy that and change it so that it counts failures and fails after N instead of failing after the first one. Test::Builder is now a compatibility layer around Test2 these days so this will count failures Test::Builder will miss. Also this plugin does not have the problem of loosing diagnostics (most of the time) that the Test::Builder override does that requires the override to wait for the next assertion before exiting. Finally this way of doing it requires no hacks or monkey-patching, instead it uses things put in place to intentionally make what you want possible.