Re: How To Test

Simply don't use weirdo testing ideas like Test::NoWarnings. This module only ever causes grief when a harmless warning pops up.

You can make testing as hard on yourself as you like. Include no tests, or a test.pl that only outputs

1..1
ok
[download]

You are quite vague about your situation, code-wise. There are no "somewhat OK" tests. Write all your tests so that they pass, fail, or skip if unapplicable. If you can't test a function that way, either skip it completely, or split it or marke it parametrizable.

As an example, I'm currently writing Win32::Wlan. This module is ugly to test, because it tries to fetch information from the Wlan surroundings. This means that I will skip tests in many situations:

Not on Windows
No Wlan API available (it starts with XP SP3 or so)

Even if these conditions pass, the subsequent tests are still split up into two parts

Tests that output diag information about the currently seen networks. I use these tests locally for manual inspection of the results.
Tests with canned results for the API calls. I use these tests to get reproducible results even when the environment changes.

Comment on Re: How To Test Select or Download Code

Replies are listed 'Best First'.
Re^2: How To Test by John M. Dlugosz (Monsignor) on May 15, 2011 at 09:25 UTC
You are quite vague about your situation, code-wise. There are no "somewhat OK" tests. Write all your tests so that they pass, fail, or skip if unapplicable. I didn't think I was vague. Function throws an exception if a particular parameter is a particular bad value: check. Exception string mentions the specific parameter in question: check. Exception carp's to the correct user-facing call: fail. The last thing will not prevent the module from being used, and should not fail an installation. It should not prevent the module from being flagged as broken during CPAN testing. But I do want to find out about it during CPAN testing, as I want to exploit its ability to test on all platforms and versions. So, what "situation" exists that I can use to determine whether or not the test runs?	[reply]
Re^3: How To Test by Corion (Patriarch) on May 15, 2011 at 09:41 UTC
You can't have both. A test either fails or passes. If it fails during CPAN testing, it will get marked as "FAIL". If it fails elsewhere, it will get marked as "FAIL". If it passes, it will get marked as "PASS". If you have a "FAIL", you will likely be sent a mail from the CPAN testers, so if you want to learn about things, that will happen. If you are testing for things that are not relevant, you will paint yourself in the corner that users can't automatically install your module with passing tests. If you think that the test is relevant, then include it. If you don't think it is relevant, skip it. If you want to run long tests, or tests that just check the integrity of your documentation, add these as "author tests" under the `xt/` directory of your distribution. I'm not sure how these tests are run by Makefile.PL / Build.PL - I would assume that you need to set some environment variable for these to be run.	[reply] [d/l]
Re^4: How To Test by John M. Dlugosz (Monsignor) on May 15, 2011 at 09:50 UTC
So in a nutshell, there is no concept of "warnings" and CPAN testing is a single "situation". For my users, tests should only test critical functionality.	[reply]