in reply to Re^3: STDERR in Test Results
in thread STDERR in Test Results

In that case may I suggest starting with Test::Warn?

I am now getting quite a few fails which are caused by the tester not having Test::Warn installed and it not being a required dependency.

Is it not bad practice to force the user to install a module just for the tests?

Replies are listed 'Best First'.
Re^5: STDERR in Test Results
by hippo (Archbishop) on Jun 25, 2023 at 11:17 UTC

    It's a good question. I'm a proponent of not making the end user install more than they need so in the general case I would agree.

    However, in this particular case we are talking about Test::Warn which at the time of writing this post has 365 direct dependents and 5381 total dependents. In other words, if you were to install any random dist from CPAN there's a decent chance that it will require Test::Warn. For such a widely-used dist I would have few qualms about adding this as a dependency of one of my dists.

    The other option, as already outlined by kcott, is to make it optional. List Test::Warn as one of the "recommended" or "suggested" dependencies. Then if it isn't installed and the user doesn't want to install it that's fine but you need the test which uses it to check for this and skip that block if Test::Warn is missing. See for example Exporter::Tiny which recommends Test::Warnings and Test::Fatal and then checks for their presence.


    🦛

      However, in this particular case we are talking about Test::Warn which at the time of writing this post has 365 direct dependents and 5381 total dependents

      It seems, quite a few testers don't have Test::Warn installed...

      Perhaps they only have core modules on their setup for testing.

        Perhaps they only have core modules on their setup for testing.

        Well, yes. How else could they test that all the dependencies have been correctly listed? The system is working as intended :-)


        🦛

        It seems, quite a few testers don't have Test::Warn installed...

        You should be able to detect that and skip the test(s).

        Very crudely, something like this should work (completely untested):

        #... use Test::More tests => 42; eval 'use Test::Warn;'; # just ignore if we can't load Test::Warn #... SKIP: { unless ($INC{'Test/Warn.pm'}) { # test if Test::Warn was loaded skip 'Test::Warn is not available', 1; } warning_like { warn "foo" } qr/foo/; # you may need to put this li +ne in a string eval because warning_like is exported by Test::Warn } # ...

        Or maybe put all warnings tests in a separate script and skip the entire script if Test::Warn is not available (again untested):

        # ... use Test::More; BEGIN { if (eval 'use Test::Warn; 1') { Test::More->import(tests => 42); } else { Test::More->import(skip_all => 'Test::Warn is not available'); # this line won't be reached, see Test::More } } warning_like { warn "foo" } qr/foo/; # ...

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re^5: STDERR in Test Results
by kcott (Archbishop) on Jun 25, 2023 at 10:22 UTC
    "Is it not bad practice to force the user to install a module just for the tests?"

    Not necessarily.

    In general, I'd expect the user to run `make test` (perhaps via cpan or other, similar utility). My Makefile.PL files typically include:

    'Test::More' => 0,

    Do note that if you're using features from later versions, you should adjust the minimum version accordingly; for instance, if you use done_testing(), you'll need:

    'Test::More' => 0.88,

    The end-user is not expected to run Author Tests. These usually require $ENV{RELEASE_TESTING} (or similar) to have a TRUE value. I don't know what you're using at the moment, but I believe I've seen Test::Pod and Test::Pod::Coverage (maybe others).

    My current practice is to list the modules needed for Author Tests in the README file; although, that's a fairly recent thing and I'm still evaluating whether to continue with it. However, I also supply a safety net. Here's one of my test files (specifically for personal use):

    $ cat 99-00_pod.t #!perl use v5.36; use Test::More; BEGIN { if (! $ENV{CPAN_AUTHOR_TEST}) { plan skip_all => 'Author test: $ENV{CPAN_AUTHOR_TEST} false.'; } } eval { use Test::Pod 1.52; 1; } or do { plan skip_all => 'Test::Pod 1.52 required.'; }; all_pod_files_ok();

    Skipping means that tests don't fail but do report what test module was missing. I only recommend this for Author Tests.

    You're welcome to use that. You'd need to make some adjustments around version numbers. Also, I use RELEASE_TESTING for $work; CPAN_AUTHOR_TEST, in my personal code, avoids conflicts.

    — Ken

      Thanks Ken,

      I'm rapidly discovering that there is so much more to testing than I ever imagined!