Pragma clash across modules: bignum and Math::BigFloat

Athanasius has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

Background.

From time to time I enjoy tackling Project Euler problems (106 solved to date). Over the years I’ve developed the following routine: I start a new solution in a separate file named, say, Problem_42.pm which begins like this:

package Problem_42;

use strict;
use warnings;
...
main() unless caller;

sub main
{
    # local timing code
    my $solution = solve();
    # print solution and time taken
}

sub solve
{
    ...
    $return solution;
}
...
1;
[download]

That way I can run the solution code directly as perl Problem_42.pm and get a nice output message, or I can call the module from a test script which gets only the solution:

# test.pl
...
use Test::More;
...
# Create %problems and populate with names and solutions keyed to prob
+lem numbers
...
for my $i (sort { $a <=> $b } keys %problems)
{
    my $module = 'Problem_' . $i;
    require $module . '.pm';

    my $time0  = time();
    my $result = ${module}->solve();
    my $time1  = time();

    printf "%3d (%2d s): ", $i, int($time1 - $time0 + 0.5);

    is
    (
        $result,
        $problems{$i}->{solution},
        $problems{$i}->{name} . ': answer'
    );

    print ' ' x 12;

    cmp_ok
    (
       ($time1 - $time0),
       '<',
        MAX_SECONDS,
        $problems{$i}->{name} . ': time'
    );
}

done_testing();
[download]

This works fine, except that sometimes when I add a new solution it affects the way the other modules perform.

In one such case I added a solution and found that a subsequent solution had speeded up considerably. I believe this is due to memory allocation: the new solution used heap memory which was still available when the subsequent module was run, and it so happened that the later module was adding to its available memory in increments, so with that memory already available the overhead of repeated allocations was substantially reduced.

But I have been operating on the assumption that by putting each solution into its own namespace (i.e., package), I have isolated it semantically from the other solutions. It now appears that this is not the case. Adding a module with a use bignum pragma causes a subsequent module to perform incorrectly, even though the pragma is in a separate package and the later module has the correct use declarations..

The problem. I’ve reduced the problem to a minimal case as follows: Create 3 files in the same directory:

# 1725_Pragma.pl

use strict;
use warnings;
use First;
use Second;
[download]

# First.pm

package First;

use strict;
use warnings;
#use bignum;

print "First\n";

1;
[download]

# Second.pm

package Second;

use strict;
use warnings;
use Math::BigFloat;

print "Second\n";

my $f = Math::BigFloat->new(2);
   $f->bsqrt();

print "$f\n";

1;
[download]

With use bignum commented out as shown above, the output is as expected:

16:38 >perl 1725_Pragma.pl
First
Second
1.41421356237309504880168872420969807857

16:38 >
[download]

<Update> And I get the same value for the square root of 2 if I just run perl Second.pm on its own. </Update>

But with use bignum uncommented, the result is simply wrong:

16:38 >perl 1725_Pragma.pl
First
Second
2

16:40 >
[download]

So, my questions are:

Is this a bug?
How can a pragma declared in one namespace affect the behaviour of code in a different (and supposedly separate) namespace?
Is there a workaround that will allow me to keep the current implementations of my two solutions and run them together in the same test script?

In relation to question 3, note that I’ve tried adding no bignum to the end of First.pm or to the beginning of Second.pm, but this had no effect. :-(

Setup.

Windows 8.1, 64-bit
Strawberry Perl versions 5.20.2, 5.22.1, and 5.24.0, all 64-bit and built with USE_64_BIT_INT
bignum versions 0.37, 0.42, and 0.42, respectively
Math::BigFloat versions 1.9993, 1.999722, and 1.999722, respectively.

Thanks,

Athanasius <°(((>< contra mundum Iustus alius egestas vitae, eros Piratica,

Comment on Pragma clash across modules: bignum and Math::BigFloat Select or Download Code

Replies are listed 'Best First'.
Re: Pragma clash across modules: bignum and Math::BigFloat by choroba (Cardinal) on Dec 06, 2016 at 09:04 UTC
Confirmed in 5.18.2 and 5.25.7 on Linux. This is weird, indeed. What's even weirder is that `warn ref $f` returns `Math::BigInt` , but constructors of Math::BigInt and Math::BigFloat are still seen as different: `warn $_->can('new') for qw( Math::BigInt Math::BigFloat );` [download] ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l] [select]
Re: Pragma clash across modules: bignum and Math::BigFloat by LanX (Saint) on Dec 06, 2016 at 13:10 UTC
Let me first say that I wouldn't do the test automation like you did. IMHO each test should be run in a separate process. At least that's what `prove` would do if you ran it on a directory with each test explicitly listed. Nevertheless you are raising a valid point! I briefly looked into bignum and Math::BigFloat and they have cross dependencies, sharing inheritance, AUTOLOADing methods and overloading operators. So I'm not surprised about the side effects of such (~~old~~) ehm well legacy modules. I'd say it's a bug but probably not an easily fixed one. You might want to run your code in the debugger, to trace which `bsqrt` -method is actually called in both cases. HTH! :) Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :) Je suis Charlie!}	[reply] [d/l]
Re: Pragma clash across modules: bignum and Math::BigFloat by Anonymous Monk on Dec 06, 2016 at 19:24 UTC
What's going on here is that `bignum` is meddling with the globals $Math::BigFloat::downgrade and $Math::BigInt::upgrade. It sets downgrade to Math::BigInt, which causes the BigFloat constructor to return BigInts instead, when they don't have a decimal point. Setting $Math::BigInt::upgrade makes BigInt::bsqrt() convert to BigFloat before taking the square root; the problem is, now BigFloat::bsqrt() is working on a copy and doesn't modify the original. `use bignum; my $f = Math::BigFloat->new('2'); print $f, "\n"; print ref($f), "\n"; print $f->bsqrt(), "\n"; print $f, "\n";` [download] output: `2 Math::BigInt 1.41421356237309504880168872420969807857 2` [download] There's really no way for `bignum` to do the right thing. It would have to muck around with those globals every time the flow of control entered or left its lexical scope. And the automatic upgrade feature was just a bad idea from the start. Solution: Don't use `bignum`, I'm afraid.	[reply] [d/l] [select]
Re^2: Pragma clash across modules: bignum and Math::BigFloat (hinthash) by LanX (Saint) on Dec 07, 2016 at 22:10 UTC
> bignum is meddling with the globals $Math::BigFloat::downgrade and $Math::BigInt::upgrade. I haven't checked in detail yet (the code is a bit obscure) but this sounds reasonable. But I disagree when you say > There's really no way for bignum to do the right thing. It would have to muck around with those globals every time the flow of control entered or left its lexical scope. And the automatic upgrade feature was just a bad idea from the start. Solution: Don't use bignum, I'm afraid. Athanasius point was not why bignum fails but why the use of pragma `bignum` is effecting a completely other package, since pragmas are guaranteed to be limited to a scope. It's rather a limitation of BigFloat and BigInt to use global variable which keep their state even if the scope is left. perlpragma shows a clear mechanism to limit effects to the scope using a pragma with the help of the hinthash `%H` . Since "bignum is just a thin wrapper around various modules of the Math::BigInt family" those modules should be modified in a way to check a delegated hinthash when called from a pragma. (N.B. they share the same authors) Hence a bug, in my humble opinion. Anyway using globals for such profound effects is never a good idea. Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :) Je suis Charlie!} update according to perlglossary#P pragma A standard module whose practical hints and suggestions are received (and possibly ignored) at compile time. Pragmas are named in all lowercase. I'm not sure if bignum qualifies as a pragma, because of it's rather runtime effects.	[reply] [d/l]
Re^3: Pragma clash across modules: bignum and Math::BigFloat (hinthash) by Anonymous Monk on Dec 07, 2016 at 22:36 UTC
Pragmas are named in all lowercase. But not everything that is named in all lowercase is a pragma. Take `use lib`, for instance. Not a pragma; it globally changes @INC. `bignum` tried to be a pragma, but gave up halfway.	[reply]
Re^4: Pragma clash across modules: bignum and Math::BigFloat (hinthash) by LanX (Saint) on Dec 07, 2016 at 23:06 UTC
Re^5: Pragma clash across modules: bignum and Math::BigFloat (hinthash) by Anonymous Monk on Dec 08, 2016 at 00:23 UTC
Some notes below your chosen depth have not been shown here
Re: Pragma clash across modules: bignum and Math::BigFloat by LanX (Saint) on Dec 08, 2016 at 13:07 UTC
> So, my questions are: > 1. Is this a bug? Yes! � > 2. How�can a pragma declared in one namespace affect the behaviour of code in a different (and supposedly separate) namespace? See `PerlPragma` - such in Perl implemented "module" pragma are normally supposed to set and check the hinthash to restrict actions to the calling scope. This is only done for bignum itself But the globals set in Math::BigFloat don't adapt to this. And `no bignum` is not setting them back. � > 3. Is there a workaround that will allow me to keep the current implementations of my two solutions and run them together in the same test script? What is a workaround what is a patch? Please try to prove that unsetting $upgrade and $downgrade solves it. * If yes, you could patch the importer of the effected modules to reset those flags or even better replace them with getters checking the hinthash. Tie::Scalar could be an option for this. In any case please file a bug report!!! A core pragma sabotaging other namespaces is a very worrying thing. Cheers Rolf _{(addicted to the Perl Programming Language and ☆☆☆☆ :) Je suis Charlie!} updates footnotes *) like indicated by AnoMonk here �) not fixing delegated global flags (import options) in its own unimport `sub unimport { $^H{bignum} = undef; # no longer in + effect overload::remove_constant('binary', '', 'float', '', 'integer'); }` [download] �) because experiencing an unpredictable distant effect from a remote sub-module in the dependency chain is not OK. Especially when caused by a CORE pragma	[reply] [d/l] [select]

update

updates

footnotes