Complex App testing w/ TAP::Harness (nesting/dividing?)

Seba has asked for the wisdom of the Perl Monks concerning the following question:

I'm writing some web based business application which tends to get too complicated to test - so I'm seeking for some perl wisdom not to reinvent the wheel ;)

Well, most of base packages are tested well in unit testing (Test::More etc.), but the point is that not everyting can be tested that way. I use some bussiness logic based on actual database data, so for some test I require to establish test database. I used to work with SQLite but now I'm forced to use some full-scale SQL with triggers and procedures.

What do I have now

Two separate sets of test, one for unit testing (pretty simple so I will skip them) and second one for use case testing which does the following:

establish test database environment - setup local postgresql instance (initdb in tmp dir etc...)
create test database - create schema with DBIx::Class->deploy
populate database with data - loading them from text or JSON files
start application server
do some basic function testing with WWW::Mechanize
do some proper full-scale user interface tests with WWW::Selenium

Whole app is built with Module::Build using TAP::Harness

I use Jenkins CI and it currently only runs unit testing via ./Build test and formats them to JUnit (TAP::Formatter::JUnit) so I can see reports online.

What do I want to have

I want to be able to run all tests in Jenkins environment...

So, what's the problem

The problem is that I decided to create two global test cases called ./t/unit.t and ./t/case.t. The first one looks like this:

use strict;
use warnings;
use TAP::Harness;
use Test::More 'no_plan';

my $tap = TAP::Harness->new(
    {   'color'           => 1,
        'verbosity'       => 1,
        'lib'             => ['lib', 'blib/lib', 'blib/arch'],
        'timer'           => 1
    }
);

my @tests = (glob('t/unit/**/*.t'), glob('t/unit/*.t'));
$tap->runtests(@tests);

ok(1, "Unit tests run");
[download]

and as you can see it just creates another TAP instance and runs all the tests in ./t/unit/ directory recursively. It runs fine, but ./Build test yells that tests are out of sequence, that they fail and so...

So my question is: Is there some other, proper way to this kind of testing

Any clues, hints or CPAN links would be greatly appreciated

Comment on Complex App testing w/ TAP::Harness (nesting/dividing?) Select or Download Code

Replies are listed 'Best First'.
Re: Complex App testing w/ TAP::Harness (nesting/dividing?) by Anonymous Monk on Jun 13, 2012 at 09:48 UTC
Rename t directory to tests so the default harness doesn't run, then from test.pl call your two independent programs, see Re^13: Do Pure Perl CPAN packages really need to use ExtUtils::Command::MM?	[reply]
Re^2: Complex App testing w/ TAP::Harness (nesting/dividing?) by Seba (Acolyte) on Jun 13, 2012 at 13:26 UTC
Your solution will probably work too, but I've found a simpler solution - less changes in current code. I just iterate over all tests on my own using TAP::Parser and now top-level TAP::Harness is happy. #!/usr/bin/perl use strict; use warnings; use TAP::Harness; use TAP::Parser; use Test::More 'no_plan'; my $tap = TAP::Harness->new( { 'color' => 1, 'verbosity' => 1, 'lib' => ['lib', 'blib/lib', 'blib/arch'], 'timer' => 1 } ); my @tests = (glob('t/unit/*/.t'), glob('t/unit/*.t')); foreach my $test (@tests) { my $parser = TAP::Parser->new({source => \$test}); while (my $result = $parser->next) { next unless $result->is_test; ok($result->is_ok, "[$test] " . $result->as_string); } } ok(1, "Unit tests run"); [download]	[reply] [d/l]