Each .t file is an independent program. How do you share something with a file on disk?

You could convert each test program you want to run for every config file into a module that will be used by a test program that looks something like:

#!/usr/bin/perl

use Test::More;

use lib ...;

my @configs      = ...;
my @test_modules = ...;

my $num_tests = 0;
for my $test_module (@test_modules) {
    require $test_module;
    $num_tests += $test_module->get_num_tests();
}

plan tests => @configs * $num_tests;

for my $config (@configs) {
    for my $test_module (@test_modules) {
        $test_module->run_tests($config);
    }
}
[download]

Thanks for the suggestions and the book reference.

Ikegami: in essence, this would be having a .t file that is actually a control file for running all tests. What I wonder right now about this approach is, how would that fit in the 'make test' generated by module-starter. 'make test' would look at all files in the t/ subdirectory. So that means putting the control file there but all other test files elsewhere outside of t/. Same goes with Narveson's suggestion.

Currently I have all tests in a single file using Test:More. That is fine, but will become kind of ugly as the project grows. This is why I'm trying out module-starter. The modules will probably never be on CPAN but, I'd nevertheless like to adopt an approach which is generalized on CPAN if only to get familiar with it. So I'd like to devise a solution that is compatible and flows nicely.

Maybe that's the solution. As Narveson remarked, might not be that clever, but will keep things going: put some 'control files' in the t/ subdir and have the actual tests in another directory.

'make test' would look at all files in the t/ subdirectory.

I'm pretty sure it only looks for .t files, so placing the modules in a subdirectory of t/ is not a requirement, but it's not a bad idea either.

how can several configuration files be used, with the same tests run for each config file ?

One way is simply to write your own loop.

for my $config_file (@LIST) {
    ok is_valid($config_file);
    # ...
    is $this, $as_expected;
}
[download]

If you have three configurations in @LIST and two tests in your loop block, then just add six to your planned test count.

This is not very clever, but it gets you going, and with test suites, it's better to install something simple right away. You can always upgrade the cleverness later.