It sounds like what you really may need is something along the lines of Test::Trait (not yet written) rather than Test::Class.

When I was writing tests to fix Sub::Uplevel, I needed to test a lot of variations of nested regular and upleveled function calls. As I had just read Higher Order Perl, I realized that my test cases could be described by a tree, so I wrote a recursive function to generate and check all my test cases. While the patch isn't incorporated yet, the problem and patch are posted on RT.

If you think your problem is really N-dimensional -- meaning that you want to test all the combinations of individual variations you described, then you might want to consider doing something similar and just traverse all the combinations.

I think the challenge will be trying to describe the test variations sufficiently orthogonally that you can easily combine them. (E.g. concurrent access can't be toggled on/off as easily as key filters.) I think you're going to want to figure that out before worrying about organizing your test files and data.

Personally, my initial thought on this is to implement the core discrete tests in a utility module. Then I would create a test script along these lines:

use Test::More;
use t::CoreTests.pm qw( run_core_tests core_test_plan );

sub generate_cases {
  # returns AoH with toggles for which variations should be used for e
+ach case
  # e.g. { filter_keys => 1, filter_values => 0, change_values => sub 
+{ } }
  #
  # adjust the complexity of cases generated to suit your needs
}

my @cases = generate_cases();

plan tests => core_test_plan() * @cases;

run_core_tests( $_ ) for @cases;
[download]

Of course, the core test utility would need to know what to do with the various toggles.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.