perltutorial
hippo
<p>One of the widely-acknowledged strengths of Perl is its use of
testing as a cornerstone. Almost every module on CPAN comes with
its own test suite and there are many modules available to help with
testing and to make it as painless as possible. For those looking to
add testing to an existing code base or looking to get started with
[wp://Test-driven_development|TDD] here is a guide to the basics.</p>
<h2>Test scripts and TAP</h2>
<p>Tests in perl are, at their heart, simple scripts which evaluate a
series of boolean expressions and output the results. No doubt you could
write such a script in a few minutes and the output would tell you how
your test criteria have fared. However, there is a standard form to the
output of the test scripts which is best adhered to. This is the
[wp://Test Anything Protocol] or TAP for short. By having your script
output the results in this format, they can be analysed by a wealth of
other programs called TAP Harnesses to provide summary data,
highlight failures, etc.</p>
<p>TAP is an open protocol but was written for Perl originally. It is
intentionally very simple. The first line of output is a range of test
numbers starting at 1 and each subsequent line consists of three fields:
a pass/fail flag which is either "ok" or "not ok", the number of the
current test and an optional description of that test. Thus a script
containing a single, passing test might output this when run:</p>
<code>
1..1
ok 1 The expected file "foo" is present
</code>
<h2>A first test script</h2>
<p>So, let's write a simple test script which will output TAP. Suppose
we want to check that nobody is running our code far in the past. Here
would be a trivial test.</p>
<code>
use strict;
use warnings;
use Time::Piece;
print "1..1\n";
print localtime(time)->year > 1999 ? 'ok' : 'not ok';
print " 1 Not in a previous century\n";
</code>
<p>When we run this without time-travelling we see this output:</p>
<code>
1..1
ok 1 Not in a previous century
</code>
<p>and we see that our test has passed.</p>
<h2>Test modules</h2>
<p>Of course there are modules on CPAN to help with testing. The
simplest of these is, appropriately enough, [metamod://Test::Simple]
which is a core module.
It is little more than a wrapper with 2 handy functions to ensure that
your TAP output is in the correct format. We can rewrite our simple
century test with this module:</p>
<code>
use strict;
use warnings;
use Time::Piece;
use Test::Simple tests => 1;
ok localtime(time)->year > 1999, 'Not in a previous century';
</code>
<p>Now there are no [doc://print] statements because the module takes
care of all the output. The <c>tests => 1</c> on line 4 sets the number
of tests we expect to run, so we no longer need to print "1..1\n"
ourselves. Similarly, the <c>ok</c> function evaluates the first
argument as a boolean expresssion and outputs the correct TAP line as a
result. The second argument is the optional description.</p>
<p>Technically it is optional but I would encourage you very strongly to
include a description for any test. If you have a script with say 50
tests in it and test 37 fails but has no description, how will you know
what is wrong? Make life easy for yourself (and your collaborators and even
the users) by describing each test in the TAP output.</p>
<h3>Other testing functions</h3>
<p>While the <c>ok</c> function is useful, the output is a simple
pass/fail - it doesn't say how it failed. If our century test fails we
don't know what year it thinks it is. For that we would need to write
more code or use code someone else has written. Fortunately there is
a plethora of other testing modules to choose from, the most common of
which is [metamod://Test::More] (also in core). This gives us a heap
of other functions so that we can easily perform different types of
evaluations and receive better feedback when they fail.</p>
<p>Let's use Test::More and its handy <c>cmp_ok</c> function in our
script.</p>
<code>
use strict;
use warnings;
use Time::Piece;
use Test::More tests => 1;
cmp_ok localtime(time)->_year, '>', 1999, 'Not in a previous century';
</code>
<p>Note that I've introduced a bug here (using <c>_year</c> instead of
<c>year</c>) so that the test will likely fail. Now our test output
looks like this:</p>
<code>
1..1
not ok 1 - Not in a previous century
# Failed test 'Not in a previous century'
# at /tmp/bar.t line 6.
# '119'
# >
# '1999'
# Looks like you failed 1 test of 1.
</code>
<p>We can see at a glance what is being tested and that the year we
actually have (119) is clearly wrong so we need to fix the bug. All
lines in TAP which start with a hash (#) are comments for the reader:
Test::More and friends use this to give us verbose reports about
how things have gone wrong.</p>
<p>There are a number of other useful comparator functions in Test::More
such as <c>is</c> for simple equality, <c>like</c> for regex and so on.
These are fully explained in the [metamod://Test::More|Test::More
documentation], but their usage is quite straightforward. Let's add a
couple of other tests to see how they are used.</p>
<code>
use strict;
use warnings;
use Time::Piece;
use Test::More tests => 3;
my $now = localtime (time);
cmp_ok $now->_year, '>', 1999, 'Not in a previous century';
is $now->time, $now->hms, 'The time() and hms() methods give the same result';
like $now->fullday, qr/day$/, 'The dayname ends in "day"';
</code>
<p>There are also control flow structures such as <c>skip</c> to avoid
running tests in certain circumstances such as an invalid underlying O/S
or absence of a particular module. We could use this here to skip the
test of the dayname if a non-English locale applies.</p>
<code>
use strict;
use warnings;
use Time::Piece 1.31_02;
use Test::More tests => 3;
Time::Piece->use_locale;
my $now = localtime (time);
cmp_ok $now->_year, '>', 1999, 'Not in a previous century';
is $now->time, $now->hms, 'The time() and hms() methods give the same result';
SKIP: {
skip 'Non-English locale', 1 unless substr ($ENV{LANG} // 'en', 0, 2) eq 'en';
like $now->fullday, qr/day$/, 'The dayname ends in "day"';
}
</code>
<p>Further still there are other modules
in the [https://www.cpan.org/modules/by-module/Test/|Test::* namespace]
to help with all manner of scenarios.</p>
<h3>Working to a plan</h3>
<p>It may be the case that the precise number of tests in the script is not
known or may change frequently. In those situations, specifying
the number of tests like <c>use Test::More tests => 3;</c> can become
unwieldy or problematic. Instead we can just <c>use Test::More;</c> and
then specify the plan later.</p>
<p>One method of doing this is to call <c>plan ()</c> as a stand-alone
statement. If the number of tests is dependent on an array which is only
computed at run time we could write</p>
<code>
plan tests => scalar @array;
</code>
<p>once the array has been populated.</p>
<p>Another approach is to use <c>done_testing (scalar @array);</c> but
as its name suggests this must only be called after the final test has
been run. The number of tests can even be omitted entirely here but that
removes the check that all the tests expected have indeed run, of course.</p>
<code>
done_testing ();
exit;
</code>
<h2>Using a harness</h2>
<p>If you have installed a module from CPAN you will probably have
noticed the test phase running. You can use the same harness on your own
test scripts by running the [metamod://prove] command. By default this
condenses the results of tests and at the end provides a summary of
which tests in which files have failed, how long the run took, etc. eg:</p>
<code>
$ prove /tmp/bar.t
/tmp/bar.t .. 1/3
# Failed test 'Not in a previous century'
# at /tmp/bar.t line 8.
# '119'
# >
# '1999'
# Looks like you failed 1 test of 3.
/tmp/bar.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/3 subtests
(less 1 skipped subtest: 1 okay)
Test Summary Report
-------------------
/tmp/bar.t (Wstat: 256 Tests: 3 Failed: 1)
Failed test: 1
Non-zero exit status: 1
Files=1, Tests=3, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.06 cusr 0.01 csys = 0.10 CPU)
Result: FAIL
</code>
<p>This is particularly useful for larger projects with many scripts/modules
each of which has many tests. If <c>prove</c> is run with no arguments it will look for
files matching <c>t/*.t</c> and run all of those in sequence.</p>
<h2>Test Driven Development</h2>
<p>Now that you can test your code you can consider [wp://Test Driven Development|TDD] as a
methodology. By writing the tests before the code you are setting out
what you expect the code to do - it's a formal representation of the
specification. Doing so is a skill in itself and many people make a
career out of being a tester.</p>
<h2>See Also</h2>
<ul>
<li>Modern Perl: [http://www.modernperlbooks.com/books/modern_perl_2016/09-managing-real-programs.html#VGVzdGluZw|Testing]</li>
<li>[https://testanything.org/]</li>
<li>[id://1154672]</li>
<li>[metamod://Test::Tutorial] for another viewpoint</li>
<li>[id://1227454] for incorporating testing into development processes</li>
<li>[id://536384]</li>
<li>The original [id://11102443|RFC] for this tutorial, including rationale and comments</li>
</ul>