in reply to Number too big to fit in integer

That's tricky: for a number that doesn't fit in UV (integer), Perl automatically stores it in NV (double) which can result in an immediate loss of precision. For my system build (5.26.1 for Ubuntu) the first integer that gets stored as a double distinguishable from ~0 is 18446744073709554600 (which has actually been stored in NV as 2^64 + 2^12):

% perl -E 'say "ok" if 18446744073709554600 > ~0'
ok
% perl -E 'say "ok" if 18446744073709554599 > ~0'
%

[download]

If that loss of precision is unacceptable for your use case, then you must avoid letting Perl do the string-to-number conversion for you. If you are writing pure Perl, Math::BigInt seems like a pretty reasonable way to sort that out. If you're writing XS code, you can call the C library's strtoul() or the perl API functions grok_number() or grok_number_flags().

Replies are listed 'Best First'.
Re^2: Number too big to fit in integer
by haukex (Archbishop) on Dec 25, 2022 at 07:53 UTC
    That's tricky: for a number that doesn't fit in UV (integer), Perl automatically stores it in NV (double) which can result in an immediate loss of precision.

    See also $Config{nv_overflows_integers_at}:

    $ uname -p
x86_64
$ perl -MConfig -le 'print eval $Config{nv_overflows_integers_at}'
9007199254740992

    [download]
[reply]
[d/l]
[select]

      I kinda like that, but eval frankly creeps me out a bit.

[reply]

        Upon further reflection, $Config{nv_overflows_integers_at} doesn't tell me anything I didn't know. It still doesn't give me something I can use to determine if an argument that looks like an integer will fit in an integer. I will give it credit for obscurity, though:-)

          DB<1> use Config

  DB<2> say $Config{nv_overflows_integers_at}
256.0*256.0*256.0*256.0*256.0*256.0*2.0*2.0*2.0*2.0*2.0

        [download]

        If I eval it, I get something that is, pretty much by definition, something that won't fit in an integer. But, as we have seen, if I compare it using > to ~0, a valid integer, it may not compare high.

[reply]
[d/l]
[select]
Re^2: Number too big to fit in integer
by syphilis (Archbishop) on Dec 31, 2022 at 13:05 UTC
    That's tricky: for a number that doesn't fit in UV (integer), Perl automatically stores it in NV (double) which can result in an immediate loss of precision. For my system build (5.26.1 for Ubuntu) the first integer that gets stored as a double distinguishable from ~0 is 18446744073709554600 (which has actually been stored in NV as 2^64 + 2^12): 
    % perl -E 'say "ok" if 18446744073709554600 > ~0'
ok
% perl -E 'say "ok" if 18446744073709554599 > ~0'
%

    [download]
     This is a bit baffling.
    With perl-5.36.0, ivsize == 8, and nvtype eq 'double', it's looking to me that the "first integer that gets stored as a double distinguishable from ~0 is" 18446744073709553665: 
    >perl -E "say 'ok' if 18446744073709553665 > ~0"
ok

>perl -E "say 'ok' if 18446744073709553664 > ~0"

>

    [download]
    Am I overlooking a bug in perl-5.26.1 ? Or is there something else going on that I've missed ?
    For me, 18446744073709554599 is correctly evaluated as being greater than ~0 : 
    >perl -E "say 'ok' if 18446744073709554599 > ~0"
ok

>

    [download]
    Cheers,
    Rob
[reply]
[d/l]
[select]

      I get the same with 5.36.0 as I did with 5.26.1. Here's what I originally used to home in on the tipping point:

      % /opt/v5.36.0/bin/perl -MMath::BigInt -wle '$z0 = Math::BigInt->new(2
+)**64; for my $i (0..12) { $zi = $z0 + 2**$i; $ni = "$zi"; $d = ($ni 
+== ~0) ? "same" : "differ"; print "$i: $d" }'
0: same
1: same
2: same
3: same
4: same
5: same
6: same
7: same
8: same
9: same
10: same
11: same
12: differ
%

      [download]

      Using eval "$zi == ~0" to make it more like direct use gives me the same result.

[reply]
[d/l]
[select]
        I get the same with 5.36.0 as I did with 5.26.1.

        Oh, I see. You were adding powers of 2 to the original value of 2**64.
        I get the same as you when I run the code you provided.

        I was incrementing by one: 
        D:\>perl -MMath::BigInt -wle "$x = Math::BigInt->new(~0); $x++; while(
+\"$x\" == ~0){$x++}; print $x;"
18446744073709553665

        [download]
        Of course, this particular perl configuration sees 18446744073709553665 and 18446744073709554600 as the same value, anyway: 
        D:\>perl -le "print 'ok' if 18446744073709553665 == 184467440737095546
+00;"
ok

        [download]
        However, that your 5.26.1 regards 18446744073709554600 and 18446744073709554599 as different values is a bug in 5.26.1. (I see the same bug in my Windows build of 5.26.0.)
        I believe that perl should regard those two values as equivalent for all IV and NV configurations. 
        D:\>perl -wle "print 'ok' if 18446744073709554600 == 18446744073709554
+599;"
ok

        [download]
        For me, that fails to output 'ok' on 5.26.0. (If 5.26.0 has been built with -Duselongdouble, then 5.26.0 does get it right and outputs 'ok'.)

        Cheers,
        Rob
[reply]
[d/l]
[select]
Re^2: Number too big to fit in integer
by jpl (Monk) on Dec 25, 2022 at 00:54 UTC

    Thanks. POSIX::strtol and POSIX::strtoul crossed my mind, too, but there are rather too many caveats for comfort.

    It isn't my use case I'm most worried about. It's the expectations of some unknown user. If high and low differ by 1, the user may be expecting to see both of them in a large sample. But if they are huge, floating point precision may collapse them into a single integer value. I'd prefer to issue a fatal message than disappoint.

    Looks like Math::BigInt may be the clunky best solution.

[reply]
[d/l]
[select]
      Why are you thinking about unsigned integers anyway? rand() is limited to the precision of a double (53 bits) and beyond that there will be integers it never generates. rand() is really only suitable for cheap throwaway random numbers anyway. Check perldoc -f rand for alternatives.
[reply]
[d/l]

In Section Seekers of Perl Wisdom