My other thought, after reading your most recent post: your benchmarks are seeming rather like the concept of "author tests", which I (and other CPAN authors I've seen) put in the 'xt' directory rather than the 't' directory. See this snippet from my already-linked Makefile.PL preamble:

# run author-tests on the original
authtest ::
    $(TEST_D) xt && $(MAKE) test TEST_FILES='xt/*.t'
[download]

You could then populate this temporary TEST_FILES value with the $h{BENCHMARK_FILES}. Or, you could still call it your bench target, but use syntax like the above to make it more like the rest of the Makefile.