Re^2: Is prove's --ignore-exit flag incomplete or ignored?

Thanks for answering, how's this:

$ cat exit.t
use strict;
use warnings;

use Test::More tests => 1;

is 1, 0, 'Tickety-boo';
exit 1;
[download]

which gives:

$ prove --failures --ignore-exit exit.t
exit.t .. 1/1 
not ok 1 - Tickety-boo

#   Failed test 'Tickety-boo'
#   at exit.t line 6.
#          got: '1'
#     expected: '0'
# Looks like your test exited with 1 just after 1.
exit.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests 

Test Summary Report
-------------------
exit.t (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=1, Tests=1,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.07 cusr 
+ 0.01 csys =  0.12 CPU)
Result: FAIL
[download]

The only difference between your code and my version is that I have a failing test as well a non-zero exit status, and with that the "Dubious" message appears. Omitting the --ignore-exit flag doesn't change my output, so it's not clear that the flag is actually doing anything at all.

Comment on Re^2: Is prove's --ignore-exit flag incomplete or ignored? Select or Download Code

Replies are listed 'Best First'.
Re^3: Is prove's --ignore-exit flag incomplete or ignored? by kcott (Archbishop) on Dec 19, 2020 at 05:36 UTC
I think you'll find that `--ignore-exit` is indeed ignoring the exit value; however, it is not ignoring failed tests. In "... our own code for generating the TAP output ...", you'll probably need to implement something along the lines of `TODO` or `SKIP` that Test::More provides which, in your code, ignores failed tests where that would be a valid thing to do. Consider these four runs: $ for i in exit_0_test_todo.t exit_1_test_todo.t; do cat $i; echo "--- +-------------"; prove --failures $i; echo "============"; prove --fai +lures --ignore-exit $i; echo "============"; done use strict; use warnings; use Test::More tests => 1; TODO: { local $TODO = 'WIP'; is 1, 0, 'Tickety-boo'; }; exit 0; ---------------- exit_0_test_todo.t .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.01 usr 0.01 sys + 0.03 cusr + 0.06 csys = 0.12 CPU) Result: PASS ============ exit_0_test_todo.t .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.02 sys + 0.03 cusr + 0.06 csys = 0.12 CPU) Result: PASS ============ use strict; use warnings; use Test::More tests => 1; TODO: { local $TODO = 'WIP'; is 1, 0, 'Tickety-boo'; }; exit 1; ---------------- exit_1_test_todo.t .. 1/1 # Looks like your test exited with 1 just af +ter 1. exit_1_test_todo.t .. Dubious, test returned 1 (wstat 256, 0x100) All 1 subtests passed Test Summary Report ------------------- exit_1_test_todo.t (Wstat: 256 Tests: 1 Failed: 0) Non-zero exit status: 1 Files=1, Tests=1, 1 wallclock secs ( 0.03 usr 0.02 sys + 0.03 cusr + 0.06 csys = 0.14 CPU) Result: FAIL ============ exit_1_test_todo.t .. 1/1 # Looks like your test exited with 1 just af +ter 1. exit_1_test_todo.t .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.06 cusr + 0.03 csys = 0.12 CPU) Result: PASS ============ [download] — Ken	[reply] [d/l] [select]
Re^4: Is prove's --ignore-exit flag incomplete or ignored? by songmaster (Beadle) on Dec 21, 2020 at 20:42 UTC
Ahhhh, my mistake, sorry guys! This is normal output from prove when at least one test fails. I wouldn't call a failing test "dubious" myself, which I assumed was some kind of an indication there's a major problem with the test program's ability to run to completion. Thanks!	[reply]