Continuing my journey towards CPAN module submission (see Preparing for my first CPAN submission).

After some pesky last minute updates/testing I've packaged my module and submitted it to CPAN (Device-CableModem-SURFboard-0.01). Things were looking pretty good. Pretty good that is until I noticed that CPAN Testers reported 17 fails (Reports for Device-CableModem-SURFboard).

The test failure isn't un-expected. The one test case in my package (the default test case created by modulemaker) attempts to create a new Device::CableModem::SURFboard. Unless you actually have a Motorola SURFboard connected, the new method will fail. It tests fine for me. I guess I just didn't expect that anyone without a cable modem (the CPAN automated testers) to also try installing and testing (my bad).

The first question I have is why I didn't receive any notice of the test fails from CPAN? The failure results look like they were emailed somewhere (that should have included me) but I've re-scanned my mail logs but there doesn't appear to be any evidence of the messages being filtered.

My second question is what to do about the test failure. I could change my package in a way to allow the new() method to succeed, modem or not, or I could modify the test case to pass whether the object is created or not. Neither option makes any more sense then simply removing the test case in the first place.

My last question is how to update CPAN with the correction. I assume that the way to do this is bump up the version number and upload again?
  • Comment on The chicken and the egg (or how do I pass the module test)

Replies are listed 'Best First'.
Re: The chicken and the egg (or how do I pass the module test)
by mr_mischief (Monsignor) on Dec 21, 2007 at 21:31 UTC
    Without regard to the code itself, I'd break the test into parts. First I'd test that I get a response from the object instantiation. Then, I'd test to see if it showed the network device attached. Based on the result of the previous test, I'd test communicating with the device or skip this test.

    It sounds, though, like your code doesn't have this kind of granularity for the test suite to test. Perhaps you should consider creating the object either way, as you already suggested. You could then have a status method for it which reports presence or absence of the device. Further future tests could be skipped if the device is not determined to be present.

[reply]
      Further tests as in the same test script? Or further tests as in additional test scripts that are conditionally skipped?

      (I need to do some reading up on Test::More)
[reply]
        Unfortunately for this discussion and probably for myself in general, I've not spent enough time with Test::More to make any implementation suggestions. I think the idea of deferring part of the tests and making the ones that require the presence of the hardware to pass conditional is sound, though.
[reply]
Re: The chicken and the egg (or how do I pass the module test)
by eserte (Deacon) on Dec 21, 2007 at 23:42 UTC
    Is your email address correctly configured at PAUSE? Maybe something is blocking mail from cpan testers on your system?

    The test failure: you can skip tests if some condition is not met. E.g. 

    my $obj = Class->new();
SKIP: {
    skip("The object could not be instantiated", $number_of_tests_to_s
+kip) if !$obj;
    # more tests here to come
}

    [download]

    And the answer for the last question is "yes".

[reply]
[d/l]
      As far as I know my email address is correctly configured at PAUSE. That is to say, I do get the PAUSE notices of uploads and indexer reports. Just not the test failures apparently. I briefly signed up on cpan-testers (anyone claiming perl is dead should spend an hour sorting through this list). But I don't think I'll try that again ;)

      I added more to the test script. Several more tests to check the results of the modem info values, and a SKIP block to ignore the tests that don't apply if there's no modem connected.

      Version 0.02 was packaged and uploaded :) I guess I'll just sit and wait for the new version to turn up and check the failure status again.
[reply]

        CPANPLUS based smoke testers will only notify the module author of a FAIL if they detect that this is the first time it has failed.

        They do this by retrieving a yml file from http://cpantesters.perl.org

        What this means in practise, is that the first CPAN Tester to report a FAIL might not send a copy of the report to the author and then subsequent testers don't either.

        The default behaviour of Test::Reporter is to send email via Net::SMTP direct to the appropriate server for 'cpan-testers@perl.org', which won't relay for authors' email addresses. ( May be will in they are a @cpan.org redirector (?) ).

        Oh well. Good work anyways.

[reply]

Back to Meditations