chargrill has asked for the wisdom of the Perl Monks concerning the following question:

Lately, I've been trying to use Devel::Cover more in my effort to prove to at least myself that I was sufficiently exercising my code with my code tests. So far, I seem to be doing a pretty good job, but have recently run into an issue where Devel::Cover suggests my coverage isn't as complete as I think it is, even though I'm pretty sure it is.

The problem seems isolated to attempting to specify a default qr// in a ->new() method, with or without a default (that is, conditionally - based on whether or not a value was passed in).

Consider the following code*:

#!/usr/bin/perl use strict; use warnings; package myObj; sub new { my $class = shift; my %args = @_; my $self = {}; $self->{ name } = $args{ name } || 'default'; $self->{ value } = $args{ value } || ''; $self->{ chars } = $args{ chars } || qr/^[a-z]+$/; return bless $self, $class; } sub set_name { my $self = shift; $self->{ name } = shift; return; } sub set_value { my $self = shift; $self->{ value } = shift; return; } sub set_chars { my $self = shift; $self->{ chars } = shift; return; } sub get_name { my $self = shift; return $self->{ name }; } sub get_value { my $self = shift; return $self->{ value }; } sub get_chars { my $self = shift; return $self->{ chars }; } package main; use Test::More tests => 11; ok( my $obj1 = myObj->new( name => 'test1', value => 'myval', chars => qr/^[0-9]$/ ), "can create a myObj specifying values" ); isa_ok( $obj1, 'myObj' ); ok( my $obj2 = myObj->new(), "can create a myObj not specifying values +" ); isa_ok( $obj2, 'myObj' ); ok( ! $obj2->set_name( 'test1' ), "can set name" ); ok( 'test1' eq $obj2->get_name(), "can get name" ); ok( ! $obj2->set_value( 'myval' ), "can set value" ); ok( 'myval' eq $obj2->get_value(), "can get value" ); ok( ! $obj2->set_chars( qr/^[0-9]$/ ), "can set chars" ); ok( qr/^[0-9]$/ eq $obj2->get_chars(), "can get chars" ); is_deeply( $obj1, $obj2, "obj1 seems deeply similar to obj2" );

Run via: perl -MDevel::Cover cover.pl, which will dutifully run the test suite (all tests successful). If you quickly eyeball the test code, it seems it exercises all the available code paths. It assigns default values if it doesn't receive values in new(), and the two resultant objects are, as far as is_deeply can tell, identical. However, if we take a closer look at the coverage report:

$ cover Reading database from /Users/kentcowgill/cover_db ---------------------------- ------ ------ ------ ------ ------ ------ + ------ File stmt bran cond sub pod time + total ---------------------------- ------ ------ ------ ------ ------ ------ + ------ cover.pl 100.0 n/a 85.7 100.0 n/a 100.0 + 98.3 Total 100.0 n/a 85.7 100.0 n/a 100.0 + 98.3 ---------------------------- ------ ------ ------ ------ ------ ------ + ------

Huh? I would think that my "condition coverage" would be 100%. Taking a look at the text of the HTML report generated:

line % coverage condition 12 100 $args{'name'} || 'default' A dec 0 0 1 1 13 100 $args{'value'} || '' A dec 0 0 1 1 14 67 $args{'chars'} || qr/^[a-z]+$/ A B dec 0 0 0 0 1 1 1 X 1

Note that the code for setting $self->{name} inside new() is the exact same as the code for setting $self->{value} and also $self->{chars}... Is there something special about a qr// that's tripping up Devel::Cover in evaluating my 'condition coverage'? Has anyone else run into this sort of thing?

* Please note that this is merely test code, heavily stripped down and even vertically condensed, to illustrate the point.



--chargrill
s**lil*; $*=join'',sort split q**; s;.*;grr; &&s+(.(.)).+$2$1+; $; = qq-$_-;s,.*,ahc,;$,.=chop for split q,,,reverse;print for($,,$;,$*,$/)

Replies are listed 'Best First'.
Re: Devel::Cover oddity with condition coverage reporting?
by shmem (Chancellor) on Oct 23, 2006 at 07:47 UTC

    It's not just qr.

    Seems like Devel::Cover can cope only with constants as RHS of an or-condition, or the other way round, an op as the RHS screws things. Running your modified code

    $self->{ name } = $args{ name } || do { 'default'}; $self->{ value } = $args{ value } || do { '' }; $self->{ chars } = $args{ chars } || do { qr{^[a-z]+$} };

    which passes fine, I get

    ---------------------------- ------ ------ ------ ------ ------ ------ + ------ File stmt bran cond sub pod time + total ---------------------------- ------ ------ ------ ------ ------ ------ + ------ cover.pl 100.0 n/a 66.7 100.0 n/a 100.0 + 95.3 Total 100.0 n/a 66.7 100.0 n/a 100.0 + 95.3 ---------------------------- ------ ------ ------ ------ ------ ------ + ------

    <update> From that output it seems that for e.g. $a || op($b) Devel::Cover erroneously adds an extra code path coverage branch for op to the conditional's to-be-covered count. You test both cases for the conditional, that makes 2 out of 3 "cases" = 66.7% coverage. </update>

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
Re: Devel::Cover oddity with condition coverage reporting?
by Khen1950fx (Canon) on Oct 23, 2006 at 09:07 UTC
    I ran your script and got the exact same results. I actually think that your script is quite good. Why? To quote the modules author, Paul Johnson:

    "Branch and condition coverage data should be mostly accurate too, although not always what one might initially expect."

    So, if you take your script and compare it with the results of other CPAN modules, the data produced by your script is very interesting. Paul Johnson ran Devel::Cover on some CPAN modules. The results might surprise you. See:

    CPAN Coverage Report

    He also wrote a paper on testing and code coverage:

    Testing and Code Coverage

    He also wrote a slide presentation to accompany the paper:

    Testing and Code Coverage-Slides