Fellow monks, I humbly submit this basic testing tutorial for review. All comments are welcomed - feel free to send a personal message instead of posting a reply if you so wish.

Rationale: There are only a couple of tutorials about testing here in the Monastery and both of them concern mocking. What is missing IMHO is a nice, gentle introduction to testing with some example code to help take out that first massive leap on the learning curve. I was late to the testing game in Perl and it's a regret that I did not get into it sooner. Hopefully this will be useful to those in a similar situation.

Updates: Add examples of is, like and skip as suggested by haukex and stevieb. Add mention of the t/ directory with prove as suggested by haukex. Add mention of done_testing() as suggested by many respondents.

Now published in the Tutorials section as Basic Testing Tutorial.

Basic Testing Tutorial

One of the widely-acknowledged strengths of Perl is its use of testing as a cornerstone. Almost every module on CPAN comes with its own test suite and there are many modules available to help with testing and to make it as painless as possible. For those looking to add testing to an existing code base or looking to get started with TDD here is a guide to the basics.

Test scripts and TAP

Tests in perl are, at their heart, simple scripts which evaluate a series of boolean expressions and output the results. No doubt you could write such a script in a few minutes and the output would tell you how your test criteria have fared. However, there is a standard form to the output of the test scripts which is best adhered to. This is the Test Anything Protocol or TAP for short. By having your script output the results in this format, they can be analysed by a wealth of other programs called TAP Harnesses to provide summary data, highlight failures, etc.

TAP is an open protocol but was written for Perl originally. It is very simple at its heart. The first line of output is a range of test numbers starting at 1 and each subsequent line consists of three fields: a pass/fail flag which is either "ok" or "not ok", the number of the current test and an optional description of that test. Thus a script containing a single, passing test might output this when run:

1..1
ok 1 The expected file "foo" is present
[download]

A first test script

So, let's write a simple test script which will output TAP. Suppose we want to check that nobody is running our code far in the past. Here would be a trivial test.

use strict;
use warnings;
use Time::Piece;

print "1..1\n";
print localtime(time)->year > 1999 ? 'ok' : 'not ok';
print " 1 Not in a previous century\n";
[download]

When we run this without time-travelling we see this output:

1..1
ok 1 Not in a previous century
[download]

and we see that our test has passed.

Test modules

Of course there are modules on CPAN to help with testing. The simplest of these is, appropriately enough, Test::Simple which is a core module. It is little more than a wrapper with 2 handy functions to ensure that your TAP output is in the correct format. We can rewrite our simple century test with this module:

use strict;
use warnings;
use Time::Piece;
use Test::Simple tests => 1;

ok localtime(time)->year > 1999, 'Not in a previous century';
[download]

Now there are no print statements because the module takes care of all the output. The tests => 1 on line 4 sets the number of tests we expect to run, so we no longer need to print "1..1\n" ourselves. Similarly, the ok function evaluates the first argument as a boolean expresssion and outputs the correct TAP line as a result. The second argument is the optional description.

Technically it is optional but I would encourage you very strongly to include a description for any test. If you have a script with say 50 tests in it and test 37 fails but has no description, how will you know what is wrong? Make life easy for yourself (and your collaborators and even the users) by describing each test in the TAP output.

Other testing functions

While the ok function is useful, the output is a simple pass/fail - it doesn't say how it failed. If our century test fails we don't know what year it thinks it is. For that we would need to write more code or use code someone else has written. Fortunately there is a plethora of other testing modules to choose from, the most common of which is Test::More (also in core). This gives us a heap of other functions so that we can easily perform different types of evaluations and receive better feedback when they fail.

Let's use Test::More and its handy cmp_ok function in our script.

use strict;
use warnings;
use Time::Piece;
use Test::More tests => 1;

cmp_ok localtime(time)->_year, '>', 1999, 'Not in a previous century';
[download]

Note that I've introduced a bug here (using _year instead of year) so that the test will likely fail. Now our test output looks like this:

1..1
not ok 1 - Not in a previous century
#   Failed test 'Not in a previous century'
#   at /tmp/bar.t line 6.
#     '119'
#         >
#     '1999'
# Looks like you failed 1 test of 1.
[download]

We can see at a glance what is being tested and that the year we actually have (119) is clearly wrong so we need to fix the bug. All lines in TAP which start with a hash (#) are comments for the reader: Test::More and friends use this to give us verbose reports about how things have gone wrong.

There are a number of other useful comparator functions in Test::More such as is for simple equality, like for regex and so on. These are fully explained in the Test::More documentation, but their usage is quite straightforward. Let's add a couple of other tests to see how they are used.

use strict;
use warnings;
use Time::Piece;
use Test::More tests => 3;

my $now = localtime (time);
cmp_ok $now->_year, '>', 1999, 'Not in a previous century';
is $now->time, $now->hms, 'The time() and hms() methods give the same 
+result';
like $now->fullday, qr/day$/, 'The dayname ends in "day"';
[download]

There are also control flow structures such as skip to avoid running tests in certain circumstances such as an invalid underlying O/S or absence of a particular module. We could use this here to skip the test of the dayname if a non-English locale applies.

use strict;
use warnings;
use Time::Piece 1.31_02;
use Test::More tests => 3;

Time::Piece->use_locale;
my $now = localtime (time);

cmp_ok $now->_year, '>', 1999, 'Not in a previous century';
is $now->time, $now->hms, 'The time() and hms() methods give the same 
+result';

SKIP: {
    skip 'Non-English locale', 1 unless substr ($ENV{LANG} // 'en', 0,
+ 2) eq 'en';
    like $now->fullday, qr/day$/, 'The dayname ends in "day"';
}
[download]

Further still there are other modules in the Test::* namespace to help with all manner of scenarios.

Working to a plan

It may be the case that the precise number of tests in the script is not known or may change frequently. In those situations, specifying the number of tests like use Test::More tests => 3; can become unwieldy or problematic. Instead we can just use Test::More; and then specify the plan later.

One method of doing this is to call plan () as a stand-alone statement. If the number of tests is dependent on an array which is only computed at run time we could write

plan tests => scalar @array;
[download]

once the array has been populated.

Another approach is to use done_testing (scalar @array); but as its name suggests this must only be called after the final test has been run. The number of tests can even be omitted entirely here but that removes the check that all the tests expected have indeed run, of course.

done_testing ();
exit;
[download]

Using a harness

If you have installed a module from CPAN you will probably have noticed the test phase running. You can use the same harness on your own test scripts by running the prove command. By default this condenses the results of tests and at the end provides a summary of which tests in which files have failed, how long the run took, etc. eg:

$ prove /tmp/bar.t
/tmp/bar.t .. 1/3 
#   Failed test 'Not in a previous century'
#   at /tmp/bar.t line 8.
#     '119'
#         >
#     '1999'
# Looks like you failed 1 test of 3.
/tmp/bar.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/3 subtests 
        (less 1 skipped subtest: 1 okay)

Test Summary Report
-------------------
/tmp/bar.t (Wstat: 256 Tests: 3 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=1, Tests=3,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.06 cusr 
+ 0.01 csys =  0.10 CPU)
Result: FAIL
[download]

This is particularly useful for larger projects with many scripts/modules each of which has many tests. If prove is run with no arguments it will look for files matching t/*.t and run all of those in sequence.

Test Driven Development

Now that you can test your code you can consider TDD as a methodology. By writing the tests before the code you are setting out what you expect the code to do - it's a formal representation of the specification. Doing so is a skill in itself and many people make a career out of being a tester.

See Also

• Modern Perl: Testing
• https://testanything.org/
• How to ask better questions using Test::More and sample data
• Test::Tutorial for another viewpoint
• [RFC] Discipulus's step by step tutorial on module creation with tests and git for incorporating testing into development processes
• Re: Winning people over to better development practises (TDD)