I'm working on a some tests for a module and while I can figure out most cases, there are a few where I'm having trouble deciding weather to skip a test or fail a test.

For example, if 1+1 = 3, I know I need to send a fail message. If I'm testing file ownership on Dos, I know I should skip. What about cases in between?

My first instinct is to be conservative and fail when in doubt. Unfortunately, a lot of perl modules fail tests even when the modules work, and a lot of users ignore failed tests and install with force. A lot of modules test well if you're installing manually, you've read the README, and you set the correct environment variables, but fail miserably when installed with CPAN. I want tests that really mean it when they fail, so that makes me think, when in doubt, skip.

What kind of guidelines do you follow when deciding those borderline cases?

Update: Okay, so it sounds like the general consensus is, if users are dumb enough to ignore the test results and force an install, that's their own fault. We shouldn't make our tests more forgiving to accomodate them.

Thanks!
-Pileofrogs

In reply to Skip Vs. Fail by pileofrogs

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.