One other option would be to include Test::More et al in your distributions t/lib and use it from there. Personally I dislike this way of doing things since you have problems when modules are updated.
I would agree for just about any module but Test::More. However, in this case I think that including Test::More with your distribution is entirely reasonable.
- Any new features in a later version are unneeded for a given test suite.
- Bugfixes will mostly result in things passing that failed with the older version, and you'll discover those in your test suite so you'll never release with them anyway.
- Bugfixes that result in things failing that previously passed would be nice to have. But think of how this happens -- people who upgrade to the latest Test::More will suddenly see your (unchanged) module start failing, which means you have at the very least a bug in your test and should issue a new release anyway. And the new release will contain the updated Test::More.
- The final reason I can think of for upgrading is security fixes, and while I can imagine Test::More (well, Test::Builder really) creating temporary files insecurely or something like that, I just don't think the risk is that high. For this reason alone, however, you should probably use the system Test::More, if it exists, in favor of the bundled one.