I know there is a CPANTesters tool.

It tests a distribution with different perl versions (+different compile options) on different platforms (+ different distros/versions).

It's great to detect:

1. incompatibility of code with certains perl versions

2. integration tests failure (when tests involve system calls that act differently on different platforms)

3. incompatibility with different versions of core modules

However it looks useless to test accuracy of CPAN dependencies versions (prereqs/build_requires in META).

For example, we test module XYZ. It requires module ABC with "version=0".

Cpan testers will install _latest_ version of ABC, if ABC is not installed, or use existing version if ABC is installed as core module, or it's a dependency of CPAN Testers itself, or by chance was installed on the box.

I wrote script for myself which tests my code with all versions of all dependencies that I use (in turn, not all possible combinations).

Result looks useful, I found couple of old versions of modules that are not compatible with my code, couple of bug in 3rd party modules prereqs specifications. Several 3rd party module versions that don't build due to a bug which was later fixed (that part is useless for me).

I also was checking BACKPAN versions. If module is on backpan it can still be installed on user machines (installed in the past), or it can be distributed with OS package manager (without CPAN).

Now I am wondering is there similar feature in CPAN testers ?

Or any other existing service ?

Should such functionality be added to CPAN testers or it's just different from CPAN testers project goal?

Replies are listed 'Best First'.
Re: Autotesting dependencies compatibility
by MidLifeXis (Monsignor) on Jun 19, 2013 at 18:33 UTC

    The thing that comes to mind immediately is that this would create an exponential explosion in testing complexity. I am not saying that the end goal is not laudable, but that the number of tests that would need to be run is the product of (well, in round numbers):

    • the number of versions for each dependent module,
    • the number of versions for each child dependency for your dependent modules,
    • and so on throughout the dependency graph.
    The best compromise I have come up with (by no means assume that I should be an authority on this) is to test at the latest, and include exceptions in your Build.PL, Makefile.PL, or META files. If you require specifying a full dependency chain, then test with a specific set of dependencies, and use the CPAN toolset to lock that configuration down in your environment.

    --MidLifeXis

[reply]

      I would exclude "the number of versions for each child dependency for your dependent modules". (2+ level dependencies).

      If XYZ depends on ABC, and ABC depends on DEF.

      XYZ should be tested with all versions of ABC.

      and ABC should be tested with all versions of DEF.

      No need to test XYZ with all versions of DEF.

      If XYZ uses DEF directly, it should list it in prereqs. (tools like CPANTS can detect this)

[reply]

        I disagree. What if your use of ABC passes a parameter to it that tickles an incompatible behavior in a particular version of DEF? Until a test is included in ABC that replicates your usage, the problem will not be uncovered in your testing scenario.

        --MidLifeXis

[reply]
Re: Autotesting dependencies compatibility
by DrHyde (Prior) on Jun 20, 2013 at 11:09 UTC
    Someone else talked about the combinatorial explosion once you have a non-trivial dependency chain, but you could use cpXXXan to get part way there. Test your code against a fresh perl and up-to-date CPAN mirror, then test against a fresh perl and a mirror from a month ago, then two months ago, three months ago, and so on. It will still take a very long time for any non-trivial dependency chain.
[reply]
      Great link, thanks for cpXXXan! but I repeat, I only was about to test 1st level dependencies (i.e what listed as prereqs and build_requires in meta)
[reply]

Back to Meditations