cpan mod test probs

perl-diddler has asked for the wisdom of the Perl Monks concerning the following question:

I made some changes in a mod in cpan, P, and upgraded the test module as well.

During devel, and in the first 1.1.0 release I found 2 probs that I tried to check for at the beginning of the test. Problem 1 is if I have PERL5OPT set in my env with a -CSA.

First workaround was to try to unset it at the beginning of the test (ENV{PERL5OPT}=undef): No effect.

Next thing I tried was checking if it was defined and if so, printing an error to STDERR and dying, but in a compile in cpan just now, it didn't die on each test (which it should have if it checked for that value -- it only died on the one test that it affected:

#   Failed test 'test mixed digit string'
#   at t/P.t line 109.
# Looks like you failed 1 test of 27.
t/P.t .. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/27 subtests 

Test Summary Report
-------------------
t/P.t (Wstat: 256 Tests: 27 Failed: 1)
  Failed test:  27
  Non-zero exit status: 1
Files=1, Tests=27,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.58 cusr
+  0.14 csys =  0.75 CPU)
Result: FAIL
Failed 1/1 test programs. 1/27 subtests failed.
[download]

Compare that to what happens if I run make test:

bin/lib/P> make test
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-MTest::Har
+ness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib
+', 'blib/arch')" t/*.t
t/P.t .. 1/28 
*****************************************
*****************************************
*****************************************
*   Please unset PERL5OPT in your ENV   *
*****************************************
*****************************************
*****************************************
# Looks like you planned 28 tests but ran 2.
# Looks like your test exited with 255 just after 2.
t/P.t .. Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 26/28 subtests 

Test Summary Report
-------------------
t/P.t (Wstat: 65280 Tests: 2 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 28 tests but ran 2.
Files=1, Tests=2,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.02 cusr 
+ 0.01 csys =  0.07 CPU)
Result: FAIL
Failed 1/1 test programs. 0/2 subtests failed.
make: *** [test_dynamic] Error 255
[download]

I.e. most or all of the tests should fail.

2nd problem has to do with a needed pre-req, 'mem.pm'. While it is in my Makefile.PL, and ends up in the META files (my meta, yml, json..). Similar prob -- in that
1) it isn't pulling in the pre-req. So, in P.t (1.1.1), I
2) checked for it, tried to run cpan to install it if it was missing, and if it couldn't -- then die with fat error message like above -- same prob, it acts like it completely circumvents my check for the pre-req, but only sometimes! (somewhere along the line, a "use 5.12.0" got inserted in the test program (not sure how or why). So instead of this check code:

BEGIN{
  eval {require mem };
  if ($@ && $@ =~ /Can't locate mem/) {
    # attempt to install via cpan
    my $inst=`cpan -i mem`;
    $inst =~ /PASS/ || do {
    die "\n*****************************************\n".
         "*****************************************\n" .
         "*****************************************\n" .
         "*   Cannot install mem via cpan: BIGFAIL*\n".
         "*****************************************\n" .
         "*****************************************\n" .
         "*****************************************\n";
     };
  }
}
[download]

I see, In 5.12 or above:

Output from '/usr/bin/make test':

PERL_DL_NONLAZY=1 /home/sand/src/perl/repoperls/installed-perls/perl/v
+5.12.5/a2da/bin/perl "-MExtUtils::Command::MM" "-MTest::Harness" "-e"
+ "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/a
+rch')" t/*.t
Can't locate mem.pm in @INC (you may need to install the mem module) (
+@INC contains: /tmp/loop_over_bdir-23652-1RDoLF/P-1.1.1-ll8YPM/blib/l
+ib /tmp/loop_over_bdir-23652-1RDoLF/P-1.1.1-ll8YPM/blib/arch /etc/per
+l /usr/local/lib/perl/5.18.1 /usr/local/share/perl/5.18.1 /usr/lib/pe
+rl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 /usr/loc
+al/lib/site_perl .) at lib/P.pm line 147.
BEGIN failed--compilation aborted at lib/P.pm line 147.

#   Failed test 'received testcase 1'
#   at t/P.t line 135.
Use of uninitialized value $rstr in pattern match (m//) at t/P.t line 
+136.

#   Failed test at t/P.t line 136.
Can't locate mem.pm in @INC (you may need to install the mem module) (
+@INC contains: /tmp/loop_over_bdir-23652-1RDoLF/P-1.1.1-ll8YPM/blib/l
+ib /tmp/loop_over_bdir-23652-1RDoLF/P-1.1.1-ll8YPM/blib/arch /etc/per
+l /usr/local/lib/perl/5.18.1 /usr/local/share/perl/5.18.1 /usr/lib/pe
+rl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 /usr/loc
+al/lib/site_perl .) at lib/P.pm line 147.
BEGIN failed--compilation aborted at lib/P.pm line 147.

#   Failed test 'received testcase 2'
#   at t/P.t line 135.
Use of uninitialized value $rstr in pattern match (m//) at t/P.t line 
+136.
....
Test Summary Report
-------------------
t/P.t (Wstat: 6400 Tests: 27 Failed: 25)
  Failed tests:  3-27
  Non-zero exit status: 25
Files=1, Tests=27,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.11 cusr
+  0.01 csys =  0.14 CPU)
Result: FAIL
Failed 1/1 test programs. 25/27 subtests failed.
make: *** [test_dynamic] Error 25

------------------------------
PREREQUISITES
------------------------------

Prerequisite modules loaded:

requires:

    Module              Need   Have    
    ------------------- ------ --------
    mem                 v0.3.2 0.3.2   
    Test::Simple        0.44   1.001002

build_requires:

    Module              Need   Have    
    ------------------- ------ --------
    mem                 v0.3.2 0.3.2   

configure_requires:

    Module              Need   Have    
    ------------------- ------ --------
    ExtUtils::MakeMaker 0      6.83_01
[download]

Say what? How can mem be missing and yet the PREREQ's claim it is loaded?

Not only did it not show the error, if it wasn't loaded, but it then claimed it WAS both "not found", yet loaded.

I dunno, but doesn't that violate causality or some quantum law or is that like the cat being both alive and not alive at the same time? ???

Very confused....
Linda, still diddling in perl...

Comment on cpan mod test probs Select or Download Code

Replies are listed 'Best First'.
Re: cpan mod test probs by tobyink (Canon) on Nov 08, 2013 at 07:42 UTC
First of all, the mem prerequisite. In your last couple of releases it's correctly listed as a dependency, so you've done your job. No need to check for its presence in your tests. If a CPAN client tries to run your tests without installing it, then it's its own damn fault that they fail. Unsetting `$ENV{PERL5OPT}` in your script will achieve very little. By the time your script starts running, the effects of the environment variable have already happened. Instead try to get your test script and module to behave correctly when Unicode features have been enabled. `use Moops; class Cow :rw { has name => (default => 'Ermintrude') }; say Cow->new->name`	[reply] [d/l]
Re^2: cpan mod test probs by perl-diddler (Chaplain) on Nov 08, 2013 at 08:45 UTC
No need to check for its presence in your tests. If a CPAN client tries to run your tests without installing it, then it's its own damn fault that they fail True, but I want to make sure they know the reason and how they can fix it. At least with explicit tests for the dep, I can fail and early-exit the tests so they know nothing ran and can see a message about 'why'...	[reply]
Re^3: cpan mod test probs by Corion (Patriarch) on Nov 08, 2013 at 09:14 UTC
See Test::More about testing, especially the section on BAIL OUT. This is especially for when your test run detects a situation where further testing is impossible.	[reply]
Re^4: cpan mod test probs by perl-diddler (Chaplain) on Nov 08, 2013 at 10:08 UTC
Re: cpan mod test probs by kcott (Archbishop) on Nov 08, 2013 at 14:46 UTC
G'day Linda, You appear to have your answer regarding `$ENV{PERL5OPT}`. [As an aside, and really more for future reference, did you consider removing the hash element with delete rather than just assigning `undef`.] For some additional information in this general area, have you seen "The CPAN Testers Wiki" and, in particular, its section "CPAN Authors FAQ". -- Ken	[reply] [d/l] [select]
Re^2: cpan mod test probs by perl-diddler (Chaplain) on Nov 08, 2013 at 17:09 UTC
Didn't really consider it in this instance. Was more focused on getting it to work first, then I worry about optimizing later, if needed. I don't really have a good answer other than to fail if -Mutf8 is set in the ENV since the harness doesn't handle it. It seems to think it is immune from utf8 being settable outside of it's documented calls and ignored "use utf8;" in the code. Looking at the code itself, the strings were being treated as latin1. If I followed it's instructions on using utf8 I ended up with the faulty latin1 bytes being re-encoded as utf8 -- a double encoding. It didn't seem to be able to deal with the case where it was already encoded. I can't report it as a bug unless I'm willing to come up with a trivial test case for it. I've reported problems with having unexpected settings in PERL5OPT before (Perl won't make properly, for example) and was told it wasn't their problem -- just don't set things in your ENV. C'est la vie.	[reply]
Re: cpan mod test probs by Anonymous Monk on Nov 08, 2013 at 01:54 UTC
Well, for starters, you should never do this `cpan -i mem` in code you release on cpan, especially in a test	[reply] [d/l]
Re^2: cpan mod test probs by perl-diddler (Chaplain) on Nov 08, 2013 at 03:44 UTC
Right. And you realize that the code that you cite should never be called under normal circumstances. So "`cpan -i mem`" is only present to compensate for extraordinary failure conditions. The test code even tries to die in lack of pre-reqs, and dying doesn't work. I tend to be a bit dense at times, but how does your advice about what I "should (or 'should never') do", help solve the problem so that such code would never have needed to have been written in the first place?	[reply] [d/l]
Re^3: cpan mod test probs by Anonymous Monk on Nov 08, 2013 at 04:11 UTC
Right. And you realize that the code that you cite should never be called under normal circumstances. The circumstances don't matter, the code should not have been pushed to cpan I tend to be a bit dense at times, but how does your advice about what I "should (or 'should never') do", help solve the problem so that such code would never have needed to have been written in the first place? It only points out another problem , it doesn't try to solve anything	[reply]