moocow has asked for the wisdom of the Perl Monks concerning the following question:

greetings monks,

I've run into a hairy situation involving module versioning, ExtUtils::MakeMaker, and cpantesters about which I'd greatly appreciate some advice.

The situation:
I have several modules on CPAN, some of which depend on others. In particular, PDL::Ngrams uses PDL::VectorValued both at runtime and compile-time macros. Both of these modules are PDL packages using (generated) XS via PDL::PP and requiring a current and common version of PDL installed (for compiling & linking).

Desiderata:

  1. I would like the PDL::VectorValued sub-packages to share the same $VERSION, declaring it once and only once. Currently, I've put the declaration in a specialized file, PDL/VectorValued/Version.pm (which doesn't itself require PDL). The only other option afaik is the PDL::PP-generated PDL/VectorValued/Utils.pm, but that can't reasonably be used to set $PDL::VectorValued::Dev::VERSION (see (3)).
  2. I want to be able to specify dependencies in e.g. PDL-Ngrams/Makefile.PL so that CPAN and ExtUtils::MakeMaker parse them correctly. At the moment, that means specifying PREREQ_PM => { ..., 'PDL::VectorValued::Version' => ... } in PDL-Ngrams/Makefile.PL
  3. I need to be able to use 1 PDL::VectorValued sub-package (PDL::VectorValued::Dev) when building PDL::VectorValued::Utils itself (obviously this doesn't require PDL::VectorValued or PDL::VectorValued::Utils), but it makes it difficult to share $VERSION without a dedicated file.

The Problem:
cpan smokers choke: systems reporting near-identical configurations aren't testing identically; compare e.g. this PASS report with this FAIL. Both systems report themselves as "perl v5.22.0, GNU/Linux, 3.16.0-4-amd64, x86_64-linux-thread-multi", both running PDL-2.014, yet the one fails with a "PDL::VectorValued::Utils needs to be recompiled against the newly installed PDL" message during testing, while the other succeeds.

The error message is quite probably correct and would be helpful to a human: I'm assuming the FAILing smoker's machine has an old version of PDL::VectorValued installed which didn't get re-compiled when PDL-2.014 was released, built, and tested on the smoker. The inconsistency didn't get caught when evaluating PDL-Ngrams/Makefile.PL because its PREREQ_PM specifies the PDL-free PDL::VectorValued::Version as the dependency target. Uploading a new version of PDL::VectorValued whenever a new PDL version is released would probably trick smokers into testing with a consistent setup, but shouldn't be necessary since in this case (as in most others to date), no PDL::VectorValued code changes were required for the new PDL release.

A (suboptimal) workaround:
After tearing out some hair and posting to the pdl-devel list, I followed a suggestion offered there to manually check for a current "safe" version of PDL::VectorValued in PDL-Ngrams/Makefile.PL myself. This results in "UNKNOWN" reports from the cpan smokers, which is preferable to "FAIL", but still pretty unsatisfying. I'm unsure whether "NA" would be any better, since this situation doesn't seem to jive with the canonical conditions for either of those categories.

The wisdom I seek:
Can anyone suggest a configuration which would allow me to realize desiderata (1)-(3) in a manner which is more palatable to CPAN smokers than the current hacks? I suppose I can live without $PDL::VectorValued::Dev::VERSION for desideratum (1) if need be, but I'd like to avoid another hack (autoconf-style global macro replacement or some such) if possible, especially if it won't fix the bogus cpantesters reports. Alternately, is there an accepted procedure for this sort of issue? I've turned up similar questions in the context of linking to third-party libraries, but this case is "only" perl (XS) modules.

marmosets,
Bryan / moocow

Replies are listed 'Best First'.
Re: cpan smokers, PREREQ_PM, and PDL dependency
by Corion (Patriarch) on Nov 19, 2015 at 15:10 UTC

    Personally, I've ditched the idea of having "one central version" and have an (identical) $VERSION in every module of a distribution. I use Perl::Version resp. the perl-reversion script there to bump the version number in all module files at once.

      UPDATE: I think I have a patch which works...

      Thanks for showing me perl-reversion! It's nice, but I can't convince it to replace the version in one of my t/*.t test files. I'm using the format straight out of Test::More:
      BEGIN { use_ok('Some::Module', 1.02) }

      I don't see any relevant RT issues open for it. I'll keep staring at the code to figure out why, but if you happen to know of a workaround, please let me know.

        I don't think it looks for arbitrary numbers. Reading through the source, it seems to look for things like VERSION followed by something that looks like a version number.

        I guess a good workaround could be to use:

        BEGIN { my $VERSION; $VERSION = 1.02; use_ok('Some::Module', 1.02); };

        ... but then, I'm not sure what the use of use_ok is anyway. The test will crash if use_ok fails, but if it fails, there is little sense to continue anyway :)

        Just tell reversion where to search:
        perl-reversion .

        If it finds too much, be more specific about the version you want to change:

        perl-reversion --current 1.02 --bump

        Or search only some directories

        perl-reversion t lib

        etc.

        ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
      thanks, that's a good tip for consistent versioning which would at least allow PREREQ_PM to specify the "real" PDL-linked dependency, not sure it would help with the compilation/linking issue though, which is hard to test (indirectly via cpan smokers) without uploading a ton of miniscule changes to CPAN.

        I don't know if this'll help at all with your wanting to do incremental testing, but I started using Travis CI for build tests prior to uploading to CPAN to let a wider range of test platforms have at it. In doing this, I minimize to the best of my ability not uploading something to the CPAN that will fail because of something stupid, making publicly available a broken module, and taxing the CPAN Testers for nothing.

        Just put your code on Github, sign up for Travis-CI, create a .travis.yml file in your distribution's root directory (example contents below), and then with every push, you get testing on a wide range of perl versions, across Mac and Linux (no Windows unfortunately... I have two Strawberry Perl VMs I test my modules on manually prior to upload to CPAN).

        This is an exact working copy/paste of a .travis.yml file from one of my modules.

        language: perl perl: - "blead" - "5.22" - "5.20" - "5.18" - "5.16" - "5.14" - "5.12" - "5.10" os: - linux before_install: - git clone git://github.com/travis-perl/helpers ~/travis-perl-helpe +rs - source ~/travis-perl-helpers/init - build-perl - perl -V - build-dist - cd $BUILD_DIR # $BUILD_DIR is set by the build-dist co +mmand install: - cpan-install --deps # installs prereqs, including recommends - cpan-install --coverage # installs coverage prereqs, if enabled before_script: - coverage-setup script: - perl Makefile.PL # or Build.PL if it exists - make # or ./Build - make test after_success: - coverage-report

        Here's an example Travis report on one of my modules. You can then drill down to each version of perl to see the test results themselves.

        oops; apologies - the anon post was me --moocow