I believe it's better in two ways:

First, a module author gets failure reports via email (if so configured). I would want those failure reports to constitute actual failures that I need to research. I don't need to look into an NA: I know what that is already (or I should). As an author, http://analysis.cpantesters.org provides more useful analysis to me if I withhold from it failures that are not interesting to me. When I look at 250 test reports and 80% of them are failures, even though I know I can ignore the non-Win32 ones, it's easier to visualize what's going on if FAIL means that there was a failure where a PASS was expected, and where NA means there was a failure that was fully expected.

Second, as a potential user... particularly as a user who is considering a given module as part of a more complex dependency chain, a cursory overview of PASSes and FAILs (rightly or wrongly) will contribute to my decision as to whether this module merits further investigation. If I see an 80% failure rate I might not bother looking further. Shame on me, I know. But it does tell me the author didn't bother to get the installation process "right" (at least where "right" means following the best practices that the CPAN testers promote). If that part is messed up, what else is? Quick judgement call: Move on to one that is apparently less quirky.

To me it comes down to accurately categorizing issues in a way that provides clarity. NA, not applicable... that's different from FAIL, fix me.

I'm not saying you're wrong. You are obviously suggesting a more thoughtful approach: Look at the failures more closely. See what's really at issue. That's smart. And we're talking about a case where the reason should be pretty obvious when someone actually looks beyond the glaring failure rate. I'm just suggesting it is nice to get an NA where not applicable really is the case, rather than a fail.

I think a common ground here is that we can both agree that simply skipping the test suite (generating a PASS) for an operating system that isn't supported is probably not ideal.


Dave

In reply to Re^7: Building Win32::GuiTest for perl 5.14 or higher (Bad tests!) by davido
in thread Building Win32::GuiTest for perl 5.14 or higher by SuicideJunkie

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.