I've always appreciated the value of including 'tests' inside code, even before it was considered 'cool' to do so. Therefore, I'd like to know what is the benefit of using any of the various recommended test modules out there, as opposed to my standard way of doing it. I've glanced at the various modules and I just don't get the point.

Here's an example of what I mean by 'standard way of doing it' ... 

### test code
    #X# <test>no arguments
    if(000){
        print NewSkeletonDoc::FillSkeletonTemplate();
        };
        #X#</test>

    #X# <test>arguments with blank values
    if(000){
        print NewSkeletonDoc::FillSkeletonTemplate({basename=>' '});
        };
        #X#</test>

    #X# <test>arguments with test values
    if(000){
        print NewSkeletonDoc::FillSkeletonTemplate(
            {
            basename=>'foo_file',
            desc=>'this is the foo desc',
            sbty=>'perl-based skeleton template'
            }
            );
        };
        #X#</test>

### init package
    package NewSkeletonDoc;
    ### ... rest of code goes here

[download]

Replies are listed 'Best First'.
Re: What is wrong with testing like a noob
by etcshadow (Priest) on Jun 26, 2004 at 20:23 UTC
    Some of the benefits of using such standard testing frameworks include:
    • Being able to run a full automatic suite of tests by kicking off a single command.
    • Being able to summarize the infomration from those tests easily, and see what if any actual issues there are.
    • Being able to easily allow other people to test your code, again, just by running a command. This is the easiest way to get your code tested on other platforms.
    • By using common tools, your stuff is going to be easier to be quickly understood by others.
    • Any body else?

    Do I lose grammar points for too much use of the passive voice, there? I've always thought that was crap, anyway. 

    ------------
:Wq
Not an editor command: Wq

    [download]
[reply]
[d/l]

      Dittos to etcshadow, but I'd like to add my two pfenning...

      There is nothing WRONG with 'testing like a noob', but testing via the Test:: series of modules can make your testing much more thorough and comprehensive.

      In my opinion, the biggest benefit of a nice suite of unit tests doesn't come during the development of a module or class, but afterwards. Having an easy-to-run set of tests that covers your code shields you from a lot of buggy side-effects that can be created when you are making changes after the initial development.

      Hanlon's Razor - "Never attribute to malice that which can be adequately explained by stupidity"
[reply]
Re: What is wrong with testing like a new user
by chromatic (Archbishop) on Jun 26, 2004 at 21:18 UTC

    How much work is it to run your tests? Can your test code interfere with your other code? How maintainable are your tests? Can you run one set of tests with a single command? Can you run the entire test suite with one command? How much work is it to see how many tests are passing and failing? Do you have to read and interpret the results yourself or does the test code do it for you?

    I can't tell any of the answers from the code you've posted, but if your code does what I think it does, I think you might see a lot of productivity gains if you spent two minutes to learn how to use Test::Simple's ok() function.

    After that, everything is easy.

[reply]
[d/l]
Re: What is wrong with testing like a noob
by zentara (Cardinal) on Jun 27, 2004 at 13:20 UTC
    I think testing like a "noob" is a good thing. It's like "real world testing". You have to admit however, that certain kinds of tests are better automated. What if you have check the results for every value between 1 and 1000?

    But "noob" testing does pick up alot of bugs that "predefined testing" dosn't get, because the "predefined tests" are usually testing for things the programmmer IS AWARE of. Whereas the "noob" testing uncovers things which were totally unexpected.

    In Tk I use noob testing all the time. I just start clicking away and entering values that some dumb_as_ might do if he was running things without reading the instructions. I find alot of "little glitches" which the programmer dosn't account for because he/she assumes the end-user is competent.

    I'm not really a human, but I play one on earth. flash japh
[reply]
      Just poking at your code is, in the long run, a wasteful and sub-optimal activity. If you are going to go to the trouble to test something once, then it would be far better to figure out how to automate it. This way, you will have not just a one-shot test, but a long living regression test, and as such your investment in testing yields not just one pay off, but residual and passive pay off.
[reply]
Re: What is wrong with testing like a noob
by CountZero (Bishop) on Jun 27, 2004 at 18:30 UTC
    One big benefit of the way the Test-modules do the testing, is that the tests are external to your scripts/modules. That way you can totally rewrite your code and have it checked by the same test-suite to see that a given input still produces the same output.

    Even better is that you can make the test-suite before you write your code (of course you must know what output follows from what input) and have your code validated en route.

    An interesting presentation of this "Extreme Programming" concept can be found here (warning: off site PDF-file)

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

[reply]
[d/l]
      Wait just a second.

      As far as I'm concerned, I WANT my tests to be right there in my code. Maybe not as the final form in which it will be used, but certainly in the "source source" version.

      That's what Test::Inline was written for. Being able to do something like the following is a god-send.
      =pod

Document method here

=begin testing

my $Test = Foo::Bar->new('blahblah');
is( $Test->whacko, 1, "'blahblah' new Foo::Bar is whacko" );

=end testing

=cut

sub new {
    ...
}

      [download]

      The simple script that comes with Test::Inline sucks out the tests, builds it a .t script, and whenever you change the name of a method or the way it works, you can just add tests right there. The test scripts recompile to reflect the change.

      I liked this so much I wrote Test::Inline::Heavy ( as yet unreleased ) to add per-method and per-module dependancy-based ordering, and scale it up to handle dozens of classes compiling down to one script per-module.

      And it's awesome when building for clients, you can easily maintain vast numbers of tests in-place, written by different people, and when you roll it out to the client you compile the tests to get a single cohesive test directory, and then remove the relevant sections of the pod to help maintain clarity for their QA/debugging people.
[reply]
[d/l]
        If it works for you, that is OK. Everyone's mileage may vary. I'd rather not have my scripts (over)loaded with in place tests, esp. not if they have to be removed later in the final version.

        CountZero

        "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

[reply]

Back to Meditations