Strange behaviour when printing certain numeric values

syphilis has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

This pertains only to those perls whose $Config{ivsize} reports 8 && whose $Config{nvtype} reports double.
Such configurations are, of course, by far the most commonly used.

The strange behaviour is best demonstrated by this one-liner:

>perl -le "for(0x1p+60, 0x1p+60, 0x1p+60) { print $_ + 1 }"
1.15292150460685e+18
1152921504606846977
1152921504606846977
[download]

The weirdness is that the output changes after the first iteration of the for() loop even though the expression being evaluated doesn't.
This anomalous behaviour is presented in (and fixed by) this PR.
So, the solution is there for me to see at https://github.com/Perl/perl5/pull/21826/commits/a6995801da59359df42c7be557f1d1971f729bc4 ... but I still don't understand why that behaviour exists in the first place, or how the provided patch addresses it.

If instead we run:

>perl -MDevel::Peek -le "for(0x1p+60, 0x1p+60, 0x1p+60) { Dump $_ + 1 
+}"
[download]

we see that the first iteration treats "$_ + 1" as an NV, and the subsequent iterations treat "$_ + 1" as an IV ... but that doesn't really answer the question, it just re-phrases it.

Any insights ?
I pride myself on having an understanding of how these perls go about dealing with the fact that the NV has less precision than the IV, but this one has me stumped.
(Thankfully, once the PR has been merged, I won't have to ponder over it ;-)

Cheers,
Rob

Comment on Strange behaviour when printing certain numeric values Select or Download Code

Replies are listed 'Best First'.
Re: Strange behaviour when printing certain numeric values by NERDVANA (Priest) on Jan 17, 2024 at 18:02 UTC
I'm going to guess (and not take the time to verify, but maybe this gives you a lead to follow) that the constant got parsed as an SVt_NV at compile time, and then some optimization re-used that same SV since it was a constant, so the loop is operating on the same underlying SV 3 times. Then at runtime when perl has to try to figure out what to do at "$_ + 1", it probably does a bunch of logic to decide whether to do floating point addition or integer addition, and that function in the PR was previously returning True to indicate that lossless conversion from NV to IV is possible, and somehow it used the NV for the calculation but cached the IV back into the SV struct. Going forward, the SV appears to be both an NV and IV and so later addition uses integer math since it is faster. But, that wasn't true because the NV already lost the low 7 bits or so of the IV, so the IV is incorrect/not accurate and should not have been cached into the SV. After the patch, the lossless conversion function will return false, will not cache the IV, and should keep using NV math on each repeat operation. It seems like maybe a better fix would be to tell the parser to record the integer value at parse time since it would fit, but I really don't know enough about the parser internals to make a recommendation like that.	[reply]
Re^2: Strange behaviour when printing certain numeric values by hv (Prior) on Jan 17, 2024 at 19:55 UTC
# with my Ubuntu system perl v5.26.1 % perl -MDevel::Peek -wle 'for(0x1p+60, 0x1p+60, 0x1p+60) { Dump($_); +print $_ + 1; Dump($_) }' SV = NV(0x558a66569f70) at 0x558a66569f88 REFCNT = 2 FLAGS = (NOK,READONLY,PROTECT,pNOK) NV = 1.15292150460685e+18 1.15292150460685e+18 SV = PVNV(0x558a665410f0) at 0x558a66569f88 REFCNT = 2 FLAGS = (NOK,READONLY,PROTECT,pIOK,pNOK) IV = 1152921504606846976 NV = 1.15292150460685e+18 PV = 0 SV = NV(0x558a66569f88) at 0x558a66569fa0 REFCNT = 2 FLAGS = (NOK,READONLY,PROTECT,pNOK) NV = 1.15292150460685e+18 1152921504606846977 SV = NV(0x558a66569f88) at 0x558a66569fa0 REFCNT = 2 FLAGS = (NOK,READONLY,PROTECT,pNOK) NV = 1.15292150460685e+18 SV = NV(0x558a6656a018) at 0x558a6656a030 REFCNT = 2 FLAGS = (NOK,READONLY,PROTECT,pNOK) NV = 1.15292150460685e+18 1152921504606846977 SV = NV(0x558a6656a018) at 0x558a6656a030 REFCNT = 2 FLAGS = (NOK,READONLY,PROTECT,pNOK) NV = 1.15292150460685e+18 [download] So a) it is a different scalar coming in each time; b) the contents of the scalar on entry to the loop are identical each time (at least as far as Devel::Peek is showing). Perversely, the first time it shows as an NV and caches an IV value, whereas subsequently it shows as an IV but does not cache an IV value. So it feels like some internal state of the interpreter is changing and affecting future calls, which would suggest a bug. My inclination would be to single step through the sprintf implementation (`Perl_sv_vcatpvfn_flags` in `sv.c`) to see where and why behaviour changes between first and second pass; if I find a confluence of time and will I'll have a go at that, since I'm pretty familiar with that code. Update: Oh, I tried changing the testcase to `Dump($_); Dump($_+1)`, which takes print out of the equation: it's `pp_hot.c: pp_add()` that needs looking at. (Not `pp.c: pp_i_add()`, which should be relevant only under `use integer`.) And the sprintf implementation wasn't relevant anyway, since we were doing a plain `print`: that was just me looking for the keys under the lamppost. Hugo	[reply] [d/l] [select]
Re^3: Strange behaviour when printing certain numeric values by syphilis (Archbishop) on Jan 18, 2024 at 02:54 UTC
So it feels like some internal state of the interpreter is changing and affecting future calls, which would suggest a bug If there is a bug, then there's the question of whether the PR fixes the bug, or merely works around it. I'll ask about it on the PR thread, mentioning that I've also raised it here. Cheers, Rob	[reply]
Re^2: Strange behaviour when printing certain numeric values by choroba (Cardinal) on Jan 17, 2024 at 19:11 UTC
> It seems like maybe a better fix would be This comment should go to the PR, too. `map{substr$_->[0],$_->[1]\|\|0,1}[\\|\|{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^ARGV,3]`	[reply] [d/l]
Re: Strange behaviour when printing certain numeric values by ikegami (Patriarch) on Jan 19, 2024 at 14:48 UTC
Someone answered this on the PR. In short, there is a scalar that's reused, but it's the one returned by `1`. It's modified to hold it's conversion to float after the first operation. $ perl -e' use feature qw( say ); use Devel::Peek; for(0x1p+60, 0x1p+60, 0x1p+60) { my $one_ref = \1; Dump($$one_ref); $_ + $$one_ref; Dump($$one_ref); say ""; } ' SV = IV(0x564e26c9a5d8) at 0x564e26c9a5e8 REFCNT = 2 FLAGS = (IOK,READONLY,PROTECT,pIOK) IV = 1 SV = PVNV(0x564e26c4c2c0) at 0x564e26c9a5e8 REFCNT = 2 FLAGS = (IOK,NOK,READONLY,PROTECT,pIOK,pNOK) IV = 1 NV = 1 PV = 0 SV = PVNV(0x564e26c4c2c0) at 0x564e26c9a5e8 REFCNT = 2 FLAGS = (IOK,NOK,READONLY,PROTECT,pIOK,pNOK) IV = 1 NV = 1 PV = 0 SV = PVNV(0x564e26c4c2c0) at 0x564e26c9a5e8 REFCNT = 2 FLAGS = (IOK,NOK,READONLY,PROTECT,pIOK,pNOK) IV = 1 NV = 1 PV = 0 SV = PVNV(0x564e26c4c2c0) at 0x564e26c9a5e8 REFCNT = 2 FLAGS = (IOK,NOK,READONLY,PROTECT,pIOK,pNOK) IV = 1 NV = 1 PV = 0 SV = PVNV(0x564e26c4c2c0) at 0x564e26c9a5e8 REFCNT = 2 FLAGS = (IOK,NOK,READONLY,PROTECT,pIOK,pNOK) IV = 1 NV = 1 PV = 0 [download] Confirmation: `$ perl -le'for(0x1p+60, 0x1p+60, 0x1p+60) { print $_ + 1 }' 1.15292150460685e+18 1152921504606846977 1152921504606846977 $ perl -le'for(0x1p+60, 0x1p+60, 0x1p+60) { print $_ + eval("1") }' 1.15292150460685e+18 1.15292150460685e+18 1.15292150460685e+18` [download]	[reply] [d/l] [select]
Re: Strange behaviour when printing certain numeric values by harangzsolt33 (Curate) on Jan 17, 2024 at 17:02 UTC
That's strange, but what does the "p" stand for at the end of a number? It looks like a hexadecimal number, but "p" is not a valid hex digit.	[reply]
Re^2: Strange behaviour when printing certain numeric values by choroba (Cardinal) on Jan 17, 2024 at 19:08 UTC
Floating point parsing has been improved. For 5.8 users something from a distant future. `map{substr$_->[0],$_->[1]\|\|0,1}[\\|\|{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^ARGV,3]`	[reply] [d/l]