$ perl t/efl_returnszero.t 
using_csv ... 
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6de8d360dc805
1..3
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6d3e8d360dc80
ok 1 - Class if /bin/true
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6d3e8d360dc8
ok 2 - Class if /bin/false
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6d3e8d360dc
ok 3 - Is not true
ok
using_json .. 
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf917
1..3
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf91
ok 1 - Class if /bin/true
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf9
ok 2 - Class if /bin/false
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf
ok 3 - Is not true
ok
All tests successful.
Files=2, Tests=6,  2 wallclock secs ( 0.03 usr  0.00 sys +  1.64 
   cusr  0.02 csys =  1.69 CPU)
Result: PASS

$ prove -v t/efl_returnszero.t 
t/efl_returnszero.t .. 
using_csv ... 
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6d3e8d360
1..3
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6d3e8d36
ok 1 - Class if /bin/true
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6d3e8d360dc8
ok 2 - Class if /bin/false
R:  ___efl_returnszero_02_efl_test_simple_csv_12ad31a0c9a6d3e8d360dc5
ok 3 - Is not true
ok
using_json .. 
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf91
1..3
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf91
ok 1 - Class if /bin/true
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf91
ok 2 - Class if /bin/false
R:  ___efl_returnszero_02_efl_test_simple_json_b3459bce7515c6887faf92
ok 3 - Is not true
ok
All tests successful.
Files=2, Tests=6,  2 wallclock secs ( 0.03 usr  0.01 sys + 
   1.58 cusr  0.02 csys =  1.64 CPU) Result: PASS
All 3 subtests passed 

# Looks good here ^^^^, but summary is not what I expect:

Test Summary Report
-------------------
t/efl_returnszero.t (Wstat: 0 Tests: 8 Failed: 5)
  Failed tests:  1-4, 8
  Parse errors: More than one plan found in TAP output
                Tests out of sequence.  Found (1) but expected (5)
                Tests out of sequence.  Found (2) but expected (6)
                Tests out of sequence.  Found (3) but expected (7)
                Bad plan.  You planned 3 tests but ran 8.
Files=1, Tests=8,  2 wallclock secs ( 0.04 usr  0.00 sys +  1.67 
   cusr  0.03 csys =  1.74 CPU)
Result: FAIL
[download]

Why does prove report 8 tests in the summary and why the out of sequence complaints? It might have something to do with the test plans because I see a complaint about more that one. How is that bad? The goal is to run two different, but related tests, each with their own plan. Code:

#!/usr/bin/perl

use strict;
use warnings;
use Carp;
use TAP::Harness;

chdir 'test/masterfiles' or croak "Cannot cd to test/masterfiles $!";

my @tests = (
   [ './promises.cf', 'using_csv' ],
   [ './promises.cf', 'using_json' ],
);

# Final test command is exec + element from test.
my $harness = TAP::Harness->new({
      exec      => [ qw( /usr/bin/cf-agent -Kf ) ],
      test_args => {
         using_csv  => [ '-D', 'efl_returnszero_csv' ],
         using_json => [ '-D', 'efl_returnszero_json' ],
      },
      verbosity => 1,
      errors => 1,
});

my $r = $harness->runtests( @tests );
[download]