Re^2: Testing: shades of grey

My understanding of TODO tests is they're all expected to fail, and what's unexpected (and reported on) is for them to pass.

I politely disagree with this. The TODO tests are ones that run, and the result is reported, but whether they pass or not is academic. Maybe they passed, and maybe they failed. We don't care -- it's just 'interesting'. :)

Eventually the stuff that was TODO gets done, and the TODO can be moved into the realm of real tests, and you move on.

Alex / talexb / Toronto

For a long time, I had a link in my .sig going to Groklaw. I heard that as of December 2024, this link is dead. Still, thanks to PJ for all your work, we owe you so much. RIP Groklaw -- 2003 to 2013.

Comment on Re^2: Testing: shades of grey Select or Download Code

Replies are listed 'Best First'.
Re^3: Testing: shades of grey by hippo (Archbishop) on Dec 23, 2024 at 09:46 UTC
Yes, but in the context of hv's original question, `prove` does not by default report TODO tests which fail but it does report TODO tests which pass albeit without counting those towards an overall pass/fail of the test file/suite. That's the relevant part, AIUI. 🦛	[reply] [d/l]
Re^4: Testing: shades of grey by LanX (Saint) on Dec 23, 2024 at 23:02 UTC
Shouldn't verbose mode show them? The OP didn't provide us with an SSCCE tho, it's hard to tell what he really needs. Cheers Rolf _{(addicted to the Perl Programming Language :) see Wikisyntax for the Monastery}	[reply]
Re^5: Testing: shades of grey by pryrt (Abbot) on Dec 23, 2024 at 23:21 UTC
Shouldn't verbose mode show them? karlgoethebier suggested it earlier in the discussion, but hv has not come back and told us results in the specific circumstances. But creating my own simple SSCCE shows that verbose does, in fact, shows TODOs. C:\tmp\perl > type t\11163309.t use 5.014; # //, strict, say, s///r use warnings; use Test::More; ok 1, "gimme"; TODO: { local $TODO = "proof of concept"; ok 0, "failing TODO"; } done_testing; C:\tmp\perl > prove t\11163309.t .. ok All tests successful. Files=1, Tests=2, 1 wallclock secs ( 0.12 usr + 0.00 sys = 0.12 CPU +) Result: PASS C:\tmp\perl > prove -v t\11163309.t .. ok 1 - gimme not ok 2 - failing TODO # TODO proof of concept # Failed (TODO) test 'failing TODO' # at t\11163309.t line 11. 1..2 ok All tests successful. Files=1, Tests=2, 0 wallclock secs ( 0.09 usr + 0.02 sys = 0.11 CPU +) Result: PASS [download] And if the TODO test is changed to a passing value, then both non-verbose and verbose indicate in the Test Summary Report that the TODO test is now passing (to tell you that it may be safe to remove the TODO-ness): C:\tmp\perl > prove t\11163309.t .. ok All tests successful. Test Summary Report ------------------- t\11163309.t (Wstat: 0 Tests: 2 Failed: 0) TODO passed: 2 Files=1, Tests=2, 0 wallclock secs ( 0.08 usr + 0.03 sys = 0.11 CPU +) Result: PASS C:\tmp\perl > prove -v t\11163309.t .. ok 1 - gimme ok 2 - passing TODO # TODO proof of concept 1..2 ok All tests successful. Test Summary Report ------------------- t\11163309.t (Wstat: 0 Tests: 2 Failed: 0) TODO passed: 2 Files=1, Tests=2, 0 wallclock secs ( 0.09 usr + 0.02 sys = 0.11 CPU +) Result: PASS [download]	[reply] [d/l] [select]