Not that happy with this solution but I will go along with it. I still need a proper benchmark procedure for running the files in benchmark/ similar to the test procedure for running the files in t/. All via the Makefile (MakeMaker I guess). Let me know if any such thing exists.

So, the solution for the time being (thanks to pryrt for the postamble part which inserts a custom target in the Makefile!):

I am going to use prove to run the benchmarks as a fake test.

prove is a command line tool provided by Test::Harness which allows specifying module search path, i.e. can add blib/ and blib/arch to search path and therefore can load the module before installation. Also important: it accepts multiple benchmark files to run from the command line. Here is some typical usage:

prove --blib blib --blib blib/arch --verbose benchmarks/02.b benchmarks/01.b

So, my postamble now looks like this :

sub MY::postamble {
    my (undef,%h) = @_;
    require Data::Dumper;
    #print STDERR Data::Dumper->Dump([\%h], [qw(mm_args{postamble})]);
    return "BENCHMARK_FILES=$h{BENCHMARK_FILES}\n"
        . <<'POSTAMBLE';
#bench :: benchmarks/*.b
bench :: $(BENCHMARK_FILES)
        prove --blib $(INST_LIB) --blib $(INST_ARCHLIB) --verbose $^
POSTAMBLE
}
[download]

The relevant bit in the WriteMakefile() hash is:

    postamble => { BENCHMARK_FILES => 'benchmarks/*.b' }

A typical benchmark file looks like this:

use strict;
use warnings;

use Our::New::Module::To::Benchmark;

use Benchmark qw/timethese cmpthese :hireswallclock/;
use Test::More;

my $num_repeats = 2;

print "$0 : benchmarks...\n";
# shamelessly ripped off App::Benchmark
cmpthese(timethese($num_repeats, {
        'blabla, repeats '.$num_repeats.':' => \&do_one_repeat
}));
plan tests => 1;
pass('benchmark : '.__FILE__);

sub     do_one_repeat {
  # this is where a run happens for 1 repeat of the benchmark
}
1;
__END__
[download]

And finally I can do:

perl Makefile.PL && make all && make test && make bench && make instal
+l
[download]

and do some eggs and toast on my hard-working CPU

PS make bench suggested by tobyink is much cooler than make benchmarks