Athanasius has asked for the wisdom of the Perl Monks concerning the following question:

Background. I’ve started using Strawberry Perl 5.22.0, and I’m trying to install tobyink’s useful P5U module. It has various dependencies, one of which fails to install: Module::Info, which fails 2/7 tests, 11/121 subtests. From the CPAN Testers Matrix it appears that Module::Info has failed almost universally across all platforms since Perl version 5.21.2. But are these failures serious or trivial? I decided to investigate.

Many of the failures are traceable to the Subroutine B::OP::parent redefined warning which appears to be a known problem. The tests fail in two test modules:

It is the single test failure in the second of these modules that is currently puzzling me.

The problem. The subtest which fails in t/n1_modules.t is this:

is_deeply( [ sort keys %mods ], [ sort qw(Cwd strict Carp) ], "Got the correct modules" );

It fails as follows:

# Failed test (t\n1_modules_required.t at line 17) # Structures begin differing at: # $got->[2] = 'Win32' # $expected->[2] = 'strict' # Looks like you failed 1 tests of 3. t\n1_modules_required.t .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/3 subtests

This failing subtest is supposed to verify that the 3 listed modules are used or required by the test module t/lib/Bar.pm. Here are the full contents of that module:

package Bar; use Cwd; use Cwd 1; use Cwd 1.00102; use Cwd 1.1.2; BEGIN { cwd(); } BEGIN { $x = 1; $x = 2; require strict; } sub my_croak { require Carp; Carp::croak(cwd, @_); } 1;

It has 24 lines. I have used a hex editor to confirm that there is nothing extra “lurking” in the file — WYSIWYG. Note that there is no mention of the Win32 module in Bar.pm.

The labyrinth. The route taken by the chain of subroutine calls from the failing test back to the source of the discrepency is a tortuous one (for ease of reading I have replaced the full paths with ellipses):

  1. The failing subtest on line 14 of t/n1_modules_required.t compares a list of modules found against a list of expected modules. Since these are the modules used or required by t/lib/Bar.pm, the expected modules are (naturally) Cwd, strict, and Carp. The modules are found via the following calls:

    my $bar = Module::Info->new_from_module( 'Bar' ); ... my %mods = $bar->modules_required;
  2. Module::Info->new_from_module calls Module::Info->_find_all_installed, which returns this object:

    bless({ dir => "...\\t\\lib", file => "...\\t\\lib\\Bar.pm", name => "Bar", safe => 0, use_version => 0, }, "Module::Info")
  3. The call to Module::Info->modules_required gets a list of modules via the following call:

    my @mods = $self->_call_B('modules_used');
  4. _call_B in turn calls my($status, @out) = $self->_call_perl($command); with $command set to:

    "-MO=Module::Info,modules_used" "...\t\lib\Bar.pm"
  5. _call_perl calls @out = `$command 2>&1`; with $command set to:

    ...\perl\bin\perl.exe "-MO=Module::Info,modules_used" "...\t\lib\Bar. +pm"

The anomalies. Now, here’s where things get weird. First, we seem to have entered an infinite regression: modules_required calls _call_B, which calls _call_perl, which (apparently) calls modules_used. But modules_used calls modules_required, so we seem to be back where we started. But there is no infinite regression, so that can’t be what’s happening. Ok, so what does the call to:

...\perl\bin\perl.exe "-MO=Module::Info,modules_used" "...\t\lib\Bar. +pm"

actually do?

Second, that last call returns the following data:

Subroutine B::OP::parent redefined at ...\blib\lib/B/BUtils.pm line 21 +7. , use Cwd () at "...\t\lib\Bar.pm" line 3 , use Cwd (1) at "...\t\lib\Bar.pm" line 4 , use Cwd (1.00102) at "...\t\lib\Bar.pm" line 5 , use Cwd (v1.1.2) at "...\t\lib\Bar.pm" line 6 , use Win32 (0.27) at "...\t\lib\Bar.pm" line 624 , require bare strict.pm at line 15 , require bare Carp.pm at line 19 , ...\t\lib\Bar.pm syntax OK

Note that the Win32 module is reported as being used on line 624 of file Bar.pm. Not only does that file contain no mention of Win32; but it’s only 24 lines long! How does the call find an unmentioned module on a non-existent line of code?

I previously installed the same version (0.35) of Module::Info under Strawberry Perl 5.20.2, with no errors.

The questions. Can anyone shed any light on any of this? Suggest a way to carry the researches forward? Explain what happened between 5.20.2 and 5.22.0 to account for this bizarre behaviour? Assure me that force installing Module::Info will be safe?

Anyway, thanks for looking, :-)

Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

Replies are listed 'Best First'.
Re: Installing Module::Info 0.35 for Strawberry Perl 5.22.0
by BrowserUk (Patriarch) on Jun 28, 2015 at 16:18 UTC
    Note that the Win32 module is reported as being used on line 624 of file Bar.pm. Not only does that file contain no mention of Win32; but it’s only 24 lines long! How does the call find an unmentioned module on a non-existent line of code?

    Win32.pm is an unusual module. It doesn't need to be loaded, to be used:

    C:\test>perl -E"say Win32::GetOSVersion();" Service Pack 16060012107681

    It is effectively demand-loaded. And that is probably the source of the problem. At some point in the process of loading the used modules, Win32 get demand loaded in order to satisfy some requirement of one of those used modules; and in the process, Win32 gets added to %INC.

    Module::Info detects the presence/addition of Win32 and throws its hands up in horror.

    If its purpose is to check that a certain set of required modules can be loaded -- a pointless test as they would be detected and clearly flagged by the standard Perl mechanisms anyway -- but if that is its purpose, it should stick to doing that. And only that.

    There is no sense in calling foul because an extra module is loaded. That is none of its business with respect to its defined purpose.

    Module::Info, like so many of the modules in the Module::* namespace is developed by *nix programmer without reference to, or concern for, anything outside that platform. Also, like so many of them, it is "testing" something that it is pointless to test. Something that if it failed, Perl's standard error reporting mechanisms would detect and report anyway; and in a far clearer and more usable fashion.

    The best solution -- and I do mean best; not simplest or most convenient -- would be to disable all the tests that are dependent upon Module::Info; and refuse to allow it on your systems. It does things it shouldn't do.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
    I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

      Hello BrowserUk,

      Thanks, that’s very helpful. I didn’t realise that Win32 is a “magic” module — that explains a great deal.

      Module::Info, like so many of the modules in the Module::* namespace is developed by *nix programmer without reference to, or concern, for anything outside that platform.

      It’s not only Module::* modules that suffer from *nix bias. I regularly come across interesting modules, only to find that they won’t install on Windows. :-( My sense is that this problem is slowly getting better (or is that just wishful thinking?)

      The best solution ... would be to disable all the tests that are dependent upon Module::Info; and refuse to allow it on your systems.

      Unfortunately, I’ve become dependent on P5U, so I think I’m going to have to let Module::Info come along for the ride. But your information makes me feel much more comfortable about the prospect of by-passing the failing tests.

      Thanks again!

      Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

        It’s not only Module::* modules that suffer from *nix bias. I regularly come across interesting modules, only to find that they won’t install on Windows.

        And there is the whole Win32::* namespace that doesn't install elsewhere.

        I think it is inevitable that there will be platform dependencies.

        I also can't blame *nix authors who never use and don't have access to Win32 for not wishing to get involved in porting stuff to that platform; I feel similarly going the other way.

        What does bug me somewhat are infrastructural modules -- Module::Build et.al -- that break on Windows because of:

        • the use of '*nix tricks', that self-respecting *nix die-hards consider unnecessary and pointless.

          Eg. having multiple files who's names are identical except for case. (Build.pl & build.pl)

          There's no benefit on *nix; and it is guaranteed to break on windows.

        • the attempts to re-create *nix shell environments on Windows, in Perl.

          Is using perl -MExtUtils::Command -e cp /some/path/somefile /someother/path/somefile everywhere really better than using copy \some\path\somefile \someother\path\somefile on one platform?

          ExtUtils::Command - utilities to replace common UNIX commands in Makef +iles etc. The module is used to replace common UNIX commands. In all cases the f +unctions work from @ARGV rather than taking arguments. This makes the +m easier to deal with in Makefiles. perl -MExtUtils::Command -e cat files... > destination perl -MExtUtils::Command -e mv source... destination perl -MExtUtils::Command -e cp source... destination perl -MExtUtils::Command -e touch files... perl -MExtUtils::Command -e rm_f files... perl -MExtUtils::Command -e rm_rf directories... perl -MExtUtils::Command -e mkpath directories... perl -MExtUtils::Command -e eqtime source destination perl -MExtUtils::Command -e test_f file perl -MExtUtils::Command -e test_d directory perl -MExtUtils::Command -e chmod mode files...

          And that justifiction...wow!

        • Attempts to be 'clever'.

          Example: I don't use the cpan shell; because rather than doing the simple, obvious thing of just using the compiler command:cl -c ... and letting the system (path) mechanism find it (the right one); or fail if it isn't accessible; they decide to do a search for the compiler in a list of "well-known places" -- none of which would be considered well-known by any windows developer -- and when they don't find the compiler in any of those places, they take it upon themselves download and install a compiler. And break my system in the process.

          That's just willful bloody vandalism in my opinion.

        • Others, that detailing the few I have has made me so hot under the collar, I better stop :)

        My sense is that this problem is slowly getting better (or is that just wishful thinking?)

        Dunno. At times in the past it has seemed that things were improving in this regard; but then the prevailing wind changes and they swing again.

        Unfortunately, I’ve become dependent on P5U,

        Your choice of course; but (from personal experience) you may come to regret becoming dependent upon tools and utilities that do not directly contribute to your applications purpose.

        That's not a criticism of p5u per se -- tobyink produces good code and maintains them well.

        But generally, there is a fine line between using useful tools when they are effective; and becoming dependent upon those useful -- but not strictly required -- tools, to the point that if they fail, you do.

        This is the same difference between a library and a framework.

        With a library, if the library fails, or fails to be maintained, or becomes unavailable; you can mock up that library yourself, even if all it does is return hard-coded, known-to-work results in the interim; and your application can continue forward.

        With a framework; if it ceases to be usable for any reason, because your application is structured to the requirements and architecture of that framework; it effectively needs to be re-written from scratch. You are locked in to the continued availability and efficacy of the framework; and worse, all of its dependencies.

        Add to that the undeniable correlation between: the authors of frameworks; and the propensity to re-use 50% of CPAN in every module they write; and you knowingly sit yourself atop a time-bomb with 57 varieties of trigger all in the hands of a bunch of irreplaceable programmers you don't know and whom are scattered to the four winds, every time you choose to type use Framework::Du::Jour; at the top of your code.

        When Moose was framework du jour for a while; I tried to caution its authors against the practices that would make it the ticking time bomb it became. To no avail.

        Just beware of installing unnecessary dependencies into your applications and processes.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
        I didn’t realise that Win32 is a “magic” module — that explains a great deal.

        Unfortunately, that's only half of the story.

        Win32 ports of Perl implemented some functions in core, but in the namespace Win32, so they were available without having to load the Win32 module, and without loading the module. Other functions were implemented only in the Win32 module and were available only after that module had been loaded.

        That's not very clean, but at least it was documented by marking all in-core functions with [CORE] in the documentation of the Win32 module.

        Now (and that's new to me, too, just reading the documentation Win32CORE and Win32), the functions marked with [CORE] are no longer in-core, they are replaced with stubs that implicitly load the Win32 module.


        I don't think the stubs are much cleaner than having the functions in core. But at least it allows to update the Win32 code independant from perl. I would have prefered extending the heuristics for "(perhaps you forgot to load "%s"?)" to detect when a [CORE] function was called without having loaded the Win32 module. This way, perl -E "say Win32::OSVersion()" would result in the error message "Undefined subroutine &Win32::OSVersion called (perhaps you forgot to load "Win32"?)  at -e line 1." instead of just "Undefined subroutine &Win32::OSVersion called at -e line 1.". But that would have caused badly written old scripts to complain or crash, and as we all know, that's a no-go for new perl versions. Well, except for given/when and the smartmatch operator.

        The difference between Win32 [CORE] and Win32 non-[CORE] functions never made sense for me. Win32::BuildNumber() is specific for ActiveState Perl and is by definition not portable, not even to other Win32 ports. All other functions could be in an external XS module, as they are mostly thin wrappers for Win32 API calls.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re: Installing Module::Info 0.35 for Strawberry Perl 5.22.0
by syphilis (Archbishop) on Jun 29, 2015 at 00:07 UTC
    I previously installed the same version (0.35) of Module::Info under Strawberry Perl 5.20.2, with no errors

    Interesting that the latest version of Module::Info passes under Strawberry 5.20.0, but not 5.22.0.
    And it's the same on Ubuntu 14.04. Module-Info-0.35 passes fine on perl-5.20.0, but fails on perl-5.22.0.

    There's some variance between the errors on the 2 systems. On Windows:
    t\Module-Info.t (Wstat: 2560 Tests: 59 Failed: 10) Failed tests: 10-13, 43-46, 58-59 Non-zero exit status: 10 t\n1_modules_required.t (Wstat: 256 Tests: 3 Failed: 1) Failed test: 1 Non-zero exit status: 1
    On Linux, there's only the one failing test script:
    t/Module-Info.t (Wstat: 2048 Tests: 59 Failed: 8) Failed tests: 10-13, 43-46 Non-zero exit status: 8
    Are these known issues ? (One would think so, but I haven't gone looking.)
    The disturbing thing is that I don't know whether the failure represents a shortcoming in Module::Info (unimportant, IMO) or a failure of some other module being used during the tests (perhaps not so unimportant).
    I'd just go ahead and force install the problem module ... and see what ensues.

    Cheers,
    Rob