in reply to A terse, one-line alternative to TAP / JUnit?

Simple idea: does prove help?

  • Comment on Re: A terse, one-line alternative to TAP / JUnit?

Replies are listed 'Best First'.
Re^2: A terse, one-line alternative to TAP / JUnit?
by Dallaylaen (Chaplain) on Sep 01, 2018 at 16:14 UTC

    No. I don't want to combine multiple messages into shorter ones. I want an individual message to be a one-liner but retain the details.

    TAP was chosen because it's a clear way to say "condition Foo holds, condition Bar doesn't and here is why", not because I'm necessarily testing something.

      Maybe you could show some examples?

        Thanks actually.

        After restating my needs it looks very clear what I need - "a human-readable terse format that supports nesting" - and that's arguably JSON. It's a matter of naming fields now...

        A short and meaningful example isn't easy to come by, but I'll try...

        #!/usr/bin/env perl # Let's pretend this is a big "enterprise" application # with lots of modules and stuff. use strict; use warnings; use POSIX qw(strftime); use Assert::Refute { on_fail => 'carp' }; # Let's pretend this real logger (syslog, log4perl, or something) $SIG{__WARN__} = sub { my $mess = shift; chomp $mess; printf STDERR "%s %s[%d] WARN %s\n", strftime( "%Y-%m-%dT%H:%M:%S", localtime ), $0, $$, $mess; }; # Let's pretend this is a big loop with convoluted logic # iterating over tons of user-supplied data for my $num (1 .. 100) { # Let's pretend we skipped a lot of processing here # and a database request that is hard to replicate in isolation # and more processing # <----- HERE!!! # Now we want to add some runtime checks on top of that # so that we know right away when our assumptions don't hold try_refute { my $report = shift; # a report object # Let's pretend we have less stupid conditions being checked # like the fact that $total == $price * $quantity + $fee # or some object actually can TO_JSON and isn't nullified # But generally here we have a runtime assertion DSL mirroring # the interface of Test::More & co # which is by far the most convenient way # to describe _behavior_ in Perl. $report->diag( "verifying number $num" ); $report->ok( $num % 17, "No number is divisible by 17 (TODO on +ly checked first ten)"); $report->isnt( $num, 42, "No number is 42" ); }; # Now at this point on_fail callback has fired if conditions are n +ot met # and the application continues business as usual # because spice must flow! }; # Let's pretend this is not the end of our Big Enterprise Application __END__

        This looks stupid, but we actually had a miscalculated price go unnoticed for a few days because a partner's JSON was changed slightly.

        So, when I run this code, I get my "log" like follows:

        2018-09-01T21:45:48 example/04-lets-pretend.pl[2438] WARN # verifying +number 17 not ok 1 - No number is divisible by 17 (TODO only checked first ten) ok 2 - No number is 42 # Looks like 1 tests out of 2 have failed 1..2 Contract failed at example/04-lets-pretend.pl line 43. 2018-09-01T21:45:48 example/04-lets-pretend.pl[2438] WARN # verifying +number 34 not ok 1 - No number is divisible by 17 (TODO only checked first ten) ok 2 - No number is 42 # Looks like 1 tests out of 2 have failed 1..2 Contract failed at example/04-lets-pretend.pl line 43. 2018-09-01T21:45:48 example/04-lets-pretend.pl[2438] WARN # verifying +number 42 ok 1 - No number is divisible by 17 (TODO only checked first ten) not ok 2 - No number is 42 # Unexpected: 42 # Looks like 1 tests out of 2 have failed 1..2 Contract failed at example/04-lets-pretend.pl line 43. 2018-09-01T21:45:48 example/04-lets-pretend.pl[2438] WARN # verifying +number 51 not ok 1 - No number is divisible by 17 (TODO only checked first ten) ok 2 - No number is 42 # Looks like 1 tests out of 2 have failed 1..2 Contract failed at example/04-lets-pretend.pl line 43. 2018-09-01T21:45:48 example/04-lets-pretend.pl[2438] WARN # verifying +number 68 not ok 1 - No number is divisible by 17 (TODO only checked first ten) ok 2 - No number is 42 # Looks like 1 tests out of 2 have failed 1..2 Contract failed at example/04-lets-pretend.pl line 43. 2018-09-01T21:45:48 example/04-lets-pretend.pl[2438] WARN # verifying +number 85 not ok 1 - No number is divisible by 17 (TODO only checked first ten) ok 2 - No number is 42 # Looks like 1 tests out of 2 have failed 1..2 Contract failed at example/04-lets-pretend.pl line 43.

        This is arguably better than having an application silently processing wrong data, but that doesn't look pretty at all.

        So here is why I want a terse format that (1) is a one-liner, (2) isn't too verbose, and (3) contains information like condition number, its name, whether it holds, and some explanation if it doesn't hold.

        So I think the title was misleading in the OP. I don't need "another unit test report format", I need "just a report format" that happens to hold the same information, but about different conditions...