saintmike has asked for the wisdom of the Perl Monks concerning the following question:

I really like the CPAN Testers, it's a tremendous help for finding out if a module is broken on platforms that I would never have the time or patience to test on.

But there's cases where I don't want to get email, because a module obviously doesn't support a certain platform. Using SKIP for these platforms in my test suite isn't really correct because I want the module to show up broken in the stats for that platform to prevent people from downloading it and be disappointed.

Is there a way to abort the test suite in a way that says "yepp, broken, but that's not going to change any time soon because the platform you're on doesn't support half of what this module requires"?

  • Comment on Silence CPAN Testers on obviously broken platforms?

Replies are listed 'Best First'.
Re: Silence CPAN Testers on obviously broken platforms?
by davido (Cardinal) on Nov 19, 2012 at 06:32 UTC

    Several ways I can think of:

    • See the CPAN testers "Author's FAQ"; there are some suggestions for how to deal with failures that result from missing libraries, or how to deal with OS platforms that you know are incompatible. If memory serves, the suggestion often is to have Makefile.PL detect incompatible environments and 'die', which will get you an NA instead of a FAIL.
    • If it's just one or two testers, get in touch with them and see if they can work out what it is about their particular setup with is failing. And if you really have exhausted all alternatives, ask them to add your module to their ignore list (that is possible for them.... usually they'll do it without you even requesting it if your module install starts consuming astronomical amounts of resources, or starts crashing their smoker). I suspect that if you ask nicely, the one or two who are a problem for you would be willing to block-list your distribution on their smoker.

    • Look at the individual smoke test reports for the failures you would like to eliminate. Is there something in common with those failures that isn't present in the ones that pass? Perhaps there's an environment variable (which will show up in the reports) that your Makefile.PL could detect, and if present, could die. If your Makefile.PL aborts, you'll get an NA from the smoke tester instead of a FAIL. You might use Config, and match against some known criteria that shows up in the failed smoke test reports. If that configuration is detected, 'die' within Makefile.PL.

    Dave

[reply]
Re: Silence CPAN Testers on obviously broken platforms?
by Tux (Canon) on Nov 19, 2012 at 07:23 UTC

    Make sure Makefile.PL (or whatever build system you use) exits with zero status

    E.g. When one needs a $UNIFY environmental variable for a module to work, add this in top of your Makefile.PL:

    # Be kind to testers, not verbose
if (exists $ENV{AUTOMATED_TESTING} and $ENV{AUTOMATED_TESTING}) {
    exists $ENV{UNIFY} or exit 0;
    }

    [download]

    Of course you can make the expression as complicated as you wish :)

    Note that the CPAN Testers Reports will show "UNKNOWN" instead of "FAIL" and the matrix will show orange instead of red.


    Enjoy, Have FUN! H.Merijn
[reply]
[d/l]
[select]
      don't you rather mean "exit with non-zero status"? zero means success. if you want to get an UNKNOWN report, you shouldn't return zero...
[reply]

        Nope, see http://wiki.cpantesters.org/wiki/CPANAuthorNotes

        exit 0 means Makefile.PL succeeded , so rating could PASS ,FAIL, or UNKNOWN, depends on subsequent steps

        exit 2 means Makefile.PL died , so rating could be FAIL or NA depending on message

        exit 0 and 'Makefile' not created , rating is UNKNOWN

        exit 0 and 'Makefile' created, rating depends 'make' and 'make test' to determine PASS or FAIL

        Now at one point the cpan testers reporting service mixed up these conventions I described, started treating NA as FAIL and UNKNOWN as NA or some such mix of results (not important now)...

        So follow CPANAuthorNotes, to avoid FAIL, exit 0 to get UNKNOWN, or Devel::AssertOS to get NA

        UNKNOWN is safer than NA, cause some Win32 modules can run on linux, and some day they might all run on linux :)

        NA is like the faulty practice of parsing user-agent-string or testing browser version details to determine if the client supports cookies and ajax

        UNKNOWN is like trying to set a cookie to see if cookies are supported or checking if document.XMLHttpRequest is available to know if ajax is supported

[reply]

        No, the FAQ explicitly states:

        "How can I stop getting FAIL reports for missing libraries or other non-Perl dependencies?"

        If you have some special dependencies and don't want to get CPAN Testers reports if a dependency is not available, just exit from the Makefile.PL or Build.PL normally (with an exit code of 0) before the Makefile or Build file is created.

        exit 0 unless some_dependency_is_met ();

        Enjoy, Have FUN! H.Merijn
[reply]

Back to Seekers of Perl Wisdom