Organizing Module Tests Sensibly

skyknight has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to strike a balance between pleasing the Romans whilst in Rome, and organizing the testing framework for a module in a way that I deem sensible for my objectives. Specifically, while h2xs gives you a skeleton that assumes a t/ subdirectory, and a collection of tests t/*.t, I would much prefer to have a tree-like hierarchy of tests. It's not readily apparent to me if there is a good way to do this that doesn't involve an odious amount of hand hacking of the skeleton proffered by h2xs. Consequently I'm wondering what experiences other people have had with this matter.

My module has a collection of class files in a tree structure. Presently, I am using Test::Unit, and for each Foo.pm class file, there is a corresponding FooTest.pm file which contains a class that derives from Test::Unit::TestCase. Within each subdirectory of the class tree, there is a file called TestSuite.pm which inherits from Test::Unit::TestSuite, and subsumes all of the *Test.pm files from its directory, as well as any TestSuite.pm files in subdirectories. The result is a tree of test suites which allow you to test any sub-tree of the module in isolation, or to test the whole thing with one command line that employs the TestRunner.pl script that comes with the Test::Unit module.

Before pushing this module out to CPAN, I would like to move from using Test::Unit to Test::More, since it seems to be the de facto standard for testing, and Test::Unit seems to be ill-maintained. I do not, however, which to squash all of my tests into one directory, poorly mimicking a tree of tests by having really long and cumbersome file names with dashes or what have you.

So, in summary, the question I am posing to my fellow monks is as follows... Is there a good way to use Test::More in a fashion that plays well with Test::Harness whilst employing a battery of tests in a tree structure as opposed to a single flat directory?

Comment on Organizing Module Tests Sensibly

Replies are listed 'Best First'.
Re: Organizing Module Tests Sensibly by chromatic (Archbishop) on Jun 25, 2004 at 17:08 UTC
Test::More doesn't care about the organization of your test files. Test::Harness doesn't care either, as long as you pass a list of files to run to its `runtests()` subroutine. Whether you're using ExtUtils::MakeMaker or Module::Build, you have the ability to specify exactly which tests to run at that stage. I usually use File::Find to look for all `.t` files beneath the `t/` directory and its subdirectories and pass that list (or catenate it into a string) to the appropriate place.	[reply] [d/l] [select]
Re: Organizing Module Tests Sensibly by water (Deacon) on Jun 25, 2004 at 17:28 UTC
Totally agree with chromatic. Keep the tests in the same directories as the modules -- makes it easier to keep them up-to-date. While File::Find is a good wrapper for a programmatic smoke suite, the "prove" command (from T::Harness? Or is it from T::More? I forget) is great for running the suite from the command line, often, and flexibly. More testing = less bugs = less work = more happiness water	[reply]
Re^2: Organizing Module Tests Sensibly by skyknight (Hermit) on Jun 25, 2004 at 17:46 UTC
Keep the tests in the same directories as the modules -- makes it easier to keep them up-to-date. That's what I've been doing thus far. I wasn't sure whether that was considered a good practice for CPAN modules. If it is in fact deemed acceptable, I'm more than happy to continue on this way. More testing = less bugs = less work = more happiness I couldn't agree more. I've been testing this module that I'm currently writing a ton along the way, and it's made it a much more pleasurable experience. It's a great feeling to perform a major refactoring of feature addition, and to run a huge battery of tests and see them all give you the green light. I guess now what it all boils down to is whether to stick with Test::Unit, or to jump over to Test::More. Test::Unit is pretty cool, but I'm concerned about it not being very well supported by the authors. The test scripts for the install break, and it hasn't been updated I think since 2001, at which point they were promising better docs. Alas, no such thing has been forthcoming and they have gone silent.	[reply]
Re^3: Organizing Module Tests Sensibly by water (Deacon) on Jun 25, 2004 at 18:58 UTC
Another nice thing about T::More is the large number of other helper modules that play nicely alongside it. I don't know if they work with T::Unit, maybe they do, but the whole suite is great. T::Exception, T::DatabaseRow, etc etc. Another link worth posting again is petdance's testing large projects with perl. And while I am on my soapbox, let me also praise Test::MockObject. test driven development -- rah! rah! rah! water,water everywhere	[reply]
Re^3: Organizing Module Tests Sensibly by lachoy (Parson) on Jun 25, 2004 at 19:21 UTC
Let's see: Test::Unit needs development help, you need Test::Unit... sounds like a match made in heaven! :-) (Isn't this how modules survive past their originators?) Chris M-x auto-bs-mode	[reply]
Re^4: Organizing Module Tests Sensibly by skyknight (Hermit) on Jun 27, 2004 at 15:12 UTC
Re: Organizing Module Tests Sensibly by hv (Prior) on Jun 26, 2004 at 01:16 UTC
It is worth mentioning that h2xs is designed to help people that don't know what they're doing. It was never intended to stop people that do know what they are doing from doing what they want to do. It is probably time for h2xs to get the same sort of refactoring that perldoc has had - to move the active code into a package in handy overridable chunks, and gut the executable to something as simple as `use Perl::H2xs; Perl::H2xs->cli(@ARGV);` Hugo	[reply] [d/l]
Re: Organizing Module Tests Sensibly by mvc (Scribe) on Jun 27, 2004 at 22:20 UTC
Give Test::Class a try. It has almost all of the Test::Unit features, yet allows you to use Test::More style assertions. With a decent runner and a way to organize your cases, it is almost perfect. Check how it was done in the latest Aspect version for an example. Basically you arrange your tests in a tree, exactly like your code. The main thing that Test::Class needs is a better runner. Something with a nice UI, more information (like what method am I testing in what class), and ability to run/rerun individual test methods.	[reply]
Re^2: Organizing Module Tests Sensibly by adrianh (Chancellor) on Jul 26, 2005 at 17:27 UTC
The main thing that Test::Class needs is a better runner. Something with a nice UI, more information (like what method am I testing in what class), and ability to run/rerun individual test methods. Most of this is on the to do list. Quite when it will get done is another matter :-)	[reply]


Do you know where your variables are?
	PerlMonks