in reply to Re^5: Developing a module, how do you do it ?
in thread Developing a module, how do you do it ?

And some of those .t file are huge and have hundreds of tests.

So the tests are "huge" yet you want to put them at the end of the module, and force perl to parse them each time the module is loaded?

It will tell you test number 271 failed. If you are lucky -- and you need to jump though more hoops to make it happen -- it might tell your in what .t file and where the failing test happened.

The Test::More/Test::Simple family of modules report a lot more than that on a test failure. At a minimum they report the test number, line number and filename where the test occurred.

The documentation for these testing modules strongly encourages you (see footnotes in "readmore" section below) to give each test a name. e.g.:

is($obj->name, "Bob", "object has correct name");

If that test fails, you get a report along the lines of:

ok 1 - object can be instantiated
not ok 2 - object has correct name
#   Failed test 'object has correct name'
#   at t/mytests.t line 12.
#          got: 'Alice'
#     expected: 'Bob'
ok 3 - object can be destroyed

This makes it pretty easy to see which test has failed, and why it's failed.


footnotes

From Test::Simple:

If you provide a $name, that will be printed along with the "ok/not ok" to make it easier to find your test when if fails (just search for the name). It also makes it easier for the next guy to understand what your test is for. It's highly recommended you use test names.

From Test::More:

All test functions take a name argument. It's optional, but highly suggested that you use it.

Replies are listed 'Best First'.
Re^7: Developing a module, how do you do it ?
by BrowserUk (Patriarch) on Mar 04, 2012 at 03:14 UTC
    So the tests are "huge" yet you want to put them at the end of the module, and force perl to parse them each time the module is loaded?

    Many of the modules on CPAN have huge .t files, because that's what the tools they use force them into writing. But I don't use tools that require me to write a dozen lines of test code to test one line of code.

    And I guarantee that, even with the tests -- which could be Autoloaded if they became a drag on performance -- not one single module of mine takes 1/1000th of the time to load that your Reprove module takes. Not one.

    ... (just search for the name) ...

    So now you've got to invent names for all your tests. Just so you can search for that name to find the test?

    That is asinine make-work.

    If I use die. It automatically "names" the test, with the file and line number.

    In a single line format that my editor (and just about any other programmers editor worthy of the name) knows how to parse right out of the box.

    And if I use Carp::cluck or Carp::confess, I get full trace-back, each line of which my (and any) editor knows how to parse.

    And if I need to add extra trace to either the tests, or the code under test, in order to track down the route to the failure, I can add it temporarily, without needing to re-write half the test suite to accommodate that temporary trace.

    Or I can use Devel::Trace to track the route to the failure; or the debugger; or Devel::Peek or ...

    And if I need to pause the test at some point -- for example, so that I can attach a (C-level) debugger -- I can just stick a <STDIN> in there.

    Ie. My test methodology allows me full access to all of the debugging tools and methods available. It doesn't force-fit me into a single one-size-fits-all methodology (ack/nack), whilst stealing my input and output, and denying me all the possibilities that entails.

    My way is infinitely better.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      So now you've got to invent names for all your tests. Just so you can search for that name to find the test?

      That is asinine make-work.

      If I use die. It automatically "names" the test, with the file and line number.

      As I have already said, Test::Simple/Test::More, etc give you the file and line number out of the box, without needing to name the test.

      But consider:

      die "reason"; # versus just die;

      If you ever provide an argument for die, you've just provided a name for a test. Is that "asinine make-work"?

      Besides which, if the line in question is in a loop, a file name and line number might not be enough - a name can be very useful to figure out what's gone wrong.

      { package Maths; sub factorial { my $n = int(pop); return $n if $n<2; $n * factorial +($n - 1) } } use Test::More; my @expected = qw/ 0 1 2 6 24 100 720 /; plan tests => scalar @expected; is(Maths::factorial($_), $expected[$_], "Factorial of $_") for 0 .. $# +expected;

      A failure on line 8 doesn't give you a clue what test has failed. A failure on line 8 named "Factorial of 5" does.

      $ perl factorial.t
      1..7
      ok 1 - Factorial of 0
      ok 2 - Factorial of 1
      ok 3 - Factorial of 2
      ok 4 - Factorial of 3
      ok 5 - Factorial of 4
      not ok 6 - Factorial of 5
      #   Failed test 'Factorial of 5'
      #   at factorial.t line 8.
      #          got: '120'
      #     expected: '100'
      ok 7 - Factorial of 6
      # Looks like you failed 1 test of 7.
      

      In this case, looking at the output, it's clear where the failure is, and checking the factorial of 5 on a calculator, it's the expected result which is in fact incorrect - my ultra-useful Maths package appears to be bug-free. Though an improvement might be to die if called with a negative number.

      { package Maths; sub factorial { my $n = int(shift); die "does not compute" if $n<0; return $n if $n<2; $n * factorial($n - 1); } } use Test::More; use Test::Exception; my @results = qw/ 0 1 2 6 24 120 720 /; for (0 .. $#results) { lives_and { is Maths::factorial($_), $results[$_] } "Factorial of +$_"; } dies_ok { Maths::factorial(-2) } "Factorial of negative number"; done_testing;
      ok 1 - Factorial of 0
      ok 2 - Factorial of 1
      ok 3 - Factorial of 2
      ok 4 - Factorial of 3
      ok 5 - Factorial of 4
      ok 6 - Factorial of 5
      ok 7 - Factorial of 6
      ok 8 - Factorial of negative number
      1..8
      

        Hm. I spent an hour responding in detail to your example. When I return, the example has changed out of all recognition. That's a losing game. Who knows what will be here when I come back, if I start over.

        Even with the turn around, you are still producing factorial( 0) == 0 when it should (by convention) be 1.

        And then what is the point of testing 0 through 7? Are you expecting Perl to forget how to do math? Or recursion? Why not test 8 through 170?

        You have no tests pertaining to 32/64-bit integer limits; or detecting when results start becoming inaccurate due to floating point round-off.

        But those are test failures, not those of the test tools.

        In vastly abbreviated summary. Nothing your test suit does cannot be done more simply and with less work, without the hierarchy of modules you use; or even with them but in the same file.

        But many of the things I routinely do whilst testing cannot be done with those modules in place.

        Ergo: those modules make work; get in the way; cut down my options and produce "results" I have no want for nor need of.

        When everything runs correctly, I want silence. (Or at the very most "All tests past".) Anything else is just noise.

        When something fails, I want to go straight to the source of the failure. I don't need to try and paraphrase the code of each test in three words of English, because once I'm at the line that failed, I can see what that line does.

        So once again, *I* have no use for those modules. They require *me* to do (at least) twice as much work to achieve the same result. That is *my* (deeply considered) opinion.

        I appreciate that your opinion is different. I'm not stopping you from using whatever cpu-sapping, productivity sapping methodologies and make-work working practices you feel you need to fit in.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

      So now you've got to invent names for all your tests. Just so you can search for that name to find the test?

      No; read the documentation, or at least the output tobyink posted.

      ... which could be Autoloaded...
      And if I use Carp::cluck or Carp::confess...
      And if I need to pause the test at some point -- for example, so that I can attach a (C-level) debugger -- I can just stick a <STDIN> in there.

      All of those things are possible with Test::Builder and friends too, without you having to edit your test files when you want to debug them. Sorry, "if" you want to debug them.

      It doesn't force-fit me into a single one-size-fits-all methodology...

      Repeating that ad nauseum doesn't make it true.

        All of those things are possible with Test::Builder

        Last month it was Test::Simple.

        Last week Test::More.

        Yesterday Test::Most.

        Today Test::Builder.

        Tomorrow? Test::Simple::Factory::Most::Builder::More.

        Next month ....?

        When the tail starts wagging the dog, it is time to dock it.

        I don't need Yet Another Layer of tooling to give me back, what I always had from the get go.

        Repeating that ad nauseum doesn't make it true.

        It is true, repeated or not.

        The fact that you are so invested in this asinine technology that you can't see it for yourself, makes repetition necessary.

        (Will you ever stick to facts, rather than resort to cheap distractions?)


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?