unintentional conversion of signaling NaN to quiet NaN

pryrt has asked for the wisdom of the Perl Monks concerning the following question:

Harkening back to my experiments to make an Expand.pm to interpret IEEE754 doubles, I found when making my tests that as I converted a signaling NaN bit string into a double then back to the bit string, it became a quiet NaN. At first I thought I just had a bug in my conversion, so then I tried BrowserUK's code from Re^2: Exploring IEEE754 floating point bit patterns., along with a new function to convert it back.

#!/usr/bin/perl

use warnings;
use strict;
use lib '.';
use Tools;  # local == Data::IEEE754::Tools

use constant {
    POS_ZERO        => '0'.'00000000000'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    POS_DENORM_1ST  => '0'.'00000000000'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000001',
    POS_DENORM_LST  => '0'.'00000000000'.'1111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',
    POS_NORM_1ST    => '0'.'00000000001'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    POS_NORM_LST    => '0'.'11111111110'.'1111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',
    POS_INF         => '0'.'11111111111'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    POS_SNAN_1ST    => '0'.'11111111111'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000001',
    POS_SNAN_LST    => '0'.'11111111111'.'0111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',
    POS_IND         => '0'.'11111111111'.'1000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    POS_QNAN_1ST    => '0'.'11111111111'.'1000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000001',
    POS_QNAN_LST    => '0'.'11111111111'.'1111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',
    NEG_ZERO        => '1'.'00000000000'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    NEG_DENORM_1ST  => '1'.'00000000000'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000001',
    NEG_DENORM_LST  => '1'.'00000000000'.'1111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',
    NEG_NORM_1ST    => '1'.'00000000001'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    NEG_NORM_LST    => '1'.'11111111110'.'1111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',
    NEG_INF         => '1'.'11111111111'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    NEG_SNAN_1ST    => '1'.'11111111111'.'0000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000001',
    NEG_SNAN_LST    => '1'.'11111111111'.'0111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',
    NEG_IND         => '1'.'11111111111'.'1000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000000',
    NEG_QNAN_1ST    => '1'.'11111111111'.'1000'.'00000000'.'00000000'.
+'00000000'.'00000000'.'00000000'.'00000001',
    NEG_QNAN_LST    => '1'.'11111111111'.'1111'.'11111111'.'11111111'.
+'11111111'.'11111111'.'11111111'.'11111111',  
};
my @list = (
    POS_ZERO, POS_DENORM_1ST, POS_DENORM_LST, POS_NORM_1ST, POS_NORM_L
+ST,
    POS_INF, POS_SNAN_1ST, POS_SNAN_LST,
    POS_IND, POS_QNAN_1ST, POS_QNAN_LST,
    NEG_ZERO, NEG_DENORM_1ST, NEG_DENORM_LST, NEG_NORM_1ST, NEG_NORM_L
+ST,
    NEG_INF, NEG_SNAN_1ST, NEG_SNAN_LST,
    NEG_IND, NEG_QNAN_1ST, NEG_QNAN_LST
);

sub bitsToDouble{ unpack 'd',  pack 'b64', scalar reverse $_[0] }
sub bitsToInts{   reverse unpack 'VV', pack 'b64', scalar reverse $_[0
+] }
sub doubleToBits{ scalar reverse unpack 'b64', pack 'd', $_[0] }
printf "%23.16g : %08x%08x >=there-and-back-again=> %08x%08x\n", 
    bitsToDouble( $_ ), bitsToInts( $_ ), 
    bitsToInts( doubleToBits( bitsToDouble( $_ ) ) )
    for @list;

print "\n";

use Test::More tests => 22;
is( doubleToBits( bitsToDouble( $_ ) ) , $_ , "bitsToDouble(doubleToBi
+ts($_))" ) for @list;
[download]

outputs

                      0 : 0000000000000000 >=there-and-back-again=> 00
+00000000000000
 4.940656458412465e-324 : 0000000000000001 >=there-and-back-again=> 00
+00000000000001
 2.225073858507201e-308 : 000fffffffffffff >=there-and-back-again=> 00
+0fffffffffffff
 2.225073858507201e-308 : 0010000000000000 >=there-and-back-again=> 00
+10000000000000
 1.797693134862316e+308 : 7fefffffffffffff >=there-and-back-again=> 7f
+efffffffffffff
                    Inf : 7ff0000000000000 >=there-and-back-again=> 7f
+f0000000000000
                    NaN : 7ff0000000000001 >=there-and-back-again=> 7f
+f8000000000001
                    NaN : 7ff7ffffffffffff >=there-and-back-again=> 7f
+ffffffffffffff
                    NaN : 7ff8000000000000 >=there-and-back-again=> 7f
+f8000000000000
                    NaN : 7ff8000000000001 >=there-and-back-again=> 7f
+f8000000000001
                    NaN : 7fffffffffffffff >=there-and-back-again=> 7f
+ffffffffffffff
                     -0 : 8000000000000000 >=there-and-back-again=> 80
+00000000000000
-4.940656458412465e-324 : 8000000000000001 >=there-and-back-again=> 80
+00000000000001
-2.225073858507201e-308 : 800fffffffffffff >=there-and-back-again=> 80
+0fffffffffffff
-2.225073858507201e-308 : 8010000000000000 >=there-and-back-again=> 80
+10000000000000
-1.797693134862316e+308 : ffefffffffffffff >=there-and-back-again=> ff
+efffffffffffff
                   -Inf : fff0000000000000 >=there-and-back-again=> ff
+f0000000000000
                    NaN : fff0000000000001 >=there-and-back-again=> ff
+f8000000000001
                    NaN : fff7ffffffffffff >=there-and-back-again=> ff
+ffffffffffffff
                    NaN : fff8000000000000 >=there-and-back-again=> ff
+f8000000000000
                    NaN : fff8000000000001 >=there-and-back-again=> ff
+f8000000000001
                    NaN : ffffffffffffffff >=there-and-back-again=> ff
+ffffffffffffff

...
not ok 7 - bitsToDouble(doubleToBits(011111111111000000000000000000000
+0000000000000000000000000000001))
#   Failed test 'bitsToDouble(doubleToBits(011111111111000000000000000
+0000000000000000000000000000000000001))'
#   at ...\buk.pl line 52.
#          got: '01111111111110000000000000000000000000000000000000000
+00000000001'
#     expected: '01111111111100000000000000000000000000000000000000000
+00000000001'
not ok 8 - bitsToDouble(doubleToBits(011111111111011111111111111111111
+1111111111111111111111111111111))
#   Failed test 'bitsToDouble(doubleToBits(011111111111011111111111111
+1111111111111111111111111111111111111))'
#   at ...\buk.pl line 52.
#          got: '01111111111111111111111111111111111111111111111111111
+11111111111'
#     expected: '01111111111101111111111111111111111111111111111111111
+11111111111'
...
not ok 18 - bitsToDouble(doubleToBits(11111111111100000000000000000000
+00000000000000000000000000000001))
#   Failed test 'bitsToDouble(doubleToBits(111111111111000000000000000
+0000000000000000000000000000000000001))'
#   at ...\buk.pl line 52.
#          got: '11111111111110000000000000000000000000000000000000000
+00000000001'
#     expected: '11111111111100000000000000000000000000000000000000000
+00000000001'
not ok 19 - bitsToDouble(doubleToBits(11111111111101111111111111111111
+11111111111111111111111111111111))
#   Failed test 'bitsToDouble(doubleToBits(111111111111011111111111111
+1111111111111111111111111111111111111))'
#   at ...\buk.pl line 52.
#          got: '11111111111111111111111111111111111111111111111111111
+11111111111'
#     expected: '11111111111101111111111111111111111111111111111111111
+11111111111'
...
[download]

... ie, it fails on just the signaling NaN, appearing like a quiet NaN bitstring.

I tried under an ancient CentOS 4.6 perl 5.8.5, and under a recent Strawberry 5.22.1 32bit, and get the same quieting of the sNaN. Am I doing something wrong, or does perl just automatically quiet signalling NaN values?

Comment on unintentional conversion of signaling NaN to quiet NaN Select or Download Code

Replies are listed 'Best First'.
Re: unintentional conversion of signaling NaN to quiet NaN by BrowserUk (Patriarch) on Jun 23, 2016 at 21:41 UTC
I think what is happening here is that the default (compiler ~~rather than~~ and therefore perl) FP exception handling mode is "No exceptions". So, any time a #SNaN value is loaded (or generated) in an XMM register, it is silently converted (SNaN \| 0008000000000000H )to a #QNaN by the FPU. But, do not take my word for it; this (FP exception handling) has to be one of the most confusing areas of modern computing. Part of the problem is that once you move away from the old x87/MMX registers to the SSE2/3/4/5/AVX et. al; where anything from 2 to 8 floating point computations are being carried out in parallel, compliance with the IEEE754 exception handling rules -- conceived long before SIMD instruction sets became the norm -- becomes very problematic. Specifically, the requirement that user exception handlers be provided with the excepting value. Reasonably simple when each FP op produces in one result; much harder when you have 2, 4 or 8 operations being performed and none, some or all could results in an exceptional value. Basically, Intel's (and AMD's) manuals suggest that if compiler writers want to provide FP exception handling and full IEEE compliance, then they would need to provide floating point emulation in the exception handler in order to reproduce the excepting value(s): E.4.1 Floating-Point Emulation Every operating system must provide a kernel level floating-point exception handler (a template was presented in Section E.2, “Software Exception Handling” above). In the following discussion, assume that a user mode floating-point exception filter is supplied for SIMD floating-point exceptions (for example as part of a library of C functions), that a user program can invoke in order to handle unmasked exceptions. The user mode floating-point exception filter (not shown here) has to be able to emulate the subset of SSE/SSE2/SSE3 instructions that can generate numeric exceptions, and has to be able to invoke a user provided floating-point exception handler for floating-point exceptions. When a floating-point exception that is not masked is raised by an SSE/SSE2/SSE3 instruction, the low-level floating-point exception handler will be called. This low-level handler may in turn call the user mode floating-point exception filter. The filter function receives the original operands of the excepting instruction as no results are provided by the hardware, whether a pre-computation or a post-computation exception has occurred. For the full skinny -- but be warned, it makes the 'What Every Computer Scientist Should Know About Floating-Point Arithmetic' paper look like a Donald Trump sound-bite; and about as useful -- see (pdf)Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 1:Basic Architecture; Appendices: C & E And I welcome your correction of my interpretation. :) With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice. Not understood.	[reply]
Re^2: unintentional conversion of signaling NaN to quiet NaN by pryrt (Abbot) on Jun 23, 2016 at 23:44 UTC
Uh-huh. As my module is focused on non-exceptioned data, and my eyes are glossed over enough as it is by your quoted paragraph, I think I'll avoid delving much deeper. :-) But it is good to know that I'm not just making a silly mistake in my translation mechanism. Thanks.	[reply]
Re: unintentional conversion of signaling NaN to quiet NaN by syphilis (Archbishop) on Jun 24, 2016 at 08:09 UTC
When I do (eg): `$d = unpack 'd', pack 'h16', scalar reverse '7ff0000000000001'` [download] I expect $d to be set to the double represented by 7ff0000000000001, not to the double represented by 7ff8000000000001. I, too, find that perl is assigning 7ff8000000000001 when it has been asked to assign 7ff0000000000001 and I regard this as a bug in perl - unless there's something in the pack/unpack documentation that allows for this unwanted behaviour. I think a perlbug report should be filed for this. Cheers, Rob	[reply] [d/l]
Re^2: unintentional conversion of signaling NaN to quiet NaN by BrowserUk (Patriarch) on Jun 24, 2016 at 09:19 UTC
I think a perlbug report should be filed for this. Thing is, there is not a lot that Perl can do about it, as it is the default (and preferred) behaviour for floating point. In the code below, the bit pattern 0x0x7ff0000000000001 displays as SNaN, only when nothing has caused the value to be loaded into a floating point register. As soon as the value is loaded into an FP register, it gets silently converted to a QNan; and if the operation causes the result to be stored back to memory, the bit pattern will have been changed to a QNaN pattern. And, I've tried every combination of Exception Mask bits in the FP control word to change that behaviour and nothing does. I do not see how Perl could change that behaviour outside of emulating FP in software? Note how both the displayed representation of the value and the bit pattern change as soon as any floating point operation is performed on the value: #include <stdio.h> #include <float.h> typedef union { double d; __int64 i; } BITS64; int main( int argc, char ** argv ) { BITS64 x; __try { _controlfp( _EM_INVALID\|_EM_DENORMAL\|_EM_ZERODIVIDE\|_EM_OVERFL +OW\|_EM_UNDERFLOW\|_EM_INEXACT, _MCW_EM ); x.i = 0x7ff0000000000001; printf( "as int: %I64u\n", x.i ); printf( "as dbl: %f\n", x.d ); x.d += 0.0; printf( "as dbl: %f\n", x.d ); printf( "as int: %I64u\n", x.i ); } __except( 1 ) { fprintf( stderr, "Exception caught!\n" ); exit( -1 ); } return 0; } /* C:\test>junk.exe as int: 9218868437227405313 as dbl: 1.#SNAN0 as dbl: 1.#QNAN0 as int: 9221120237041090561 */ [download] With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice. Not understood.	[reply] [d/l]
Re^3: unintentional conversion of signaling NaN to quiet NaN by syphilis (Archbishop) on Jun 24, 2016 at 11:35 UTC
As soon as the value is loaded into an FP register, it gets silently converted to a QNan I was thinking that when we do: `$packed = pack 'd', '7ff0000000000001'; $d = unpack 'h16', $packed;` [download] there would be no loading of the double into an FP register. But I guess that thinking must be wrong as $d is definitely set to '7ff8000000000001', on my Windows box at least. On my Linux box (also little-endian), however, there's no such problem and I can assign '7ff0000000000001' just fine. Is this difference a Windows v Linux thing ? ... or is it to be explained in terms of different hardware ? Do you know of any way that one can assign the value '7ff0000000000001' on Windows ? (I haven't found a way, yet.) Here's a nice Perl demo (utilising the union you provided) of the Windows/Linux discrepancy: use warnings; use strict; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'EOC'; #ifdef _WIN32 typedef __int64 BIGGUN; #else /* "long" is 64-bit on my linux box / typedef long BIGGUN; #endif typedef union { double d; BIGGUN i; } BITS64; void bytes(double d) { int i; void p = &d; for(i = 7; i >=0; i--) printf("%02x", ((unsigned char*)p)[i]); printf("\n"); } double get_snan () { BITS64 x; x.i = 0x7ff0000000000001; bytes(x.d); return x.d; } EOC print doubleToHex(get_snan()); sub doubleToHex { scalar reverse unpack 'h16', pack 'd', $_[0] } [download] On Windows it outputs: 7ff0000000000001 7ff8000000000001 On Ubuntu it outputs: 7ff0000000000001 7ff0000000000001 Cheers, Rob	[reply] [d/l] [select]
Re^4: unintentional conversion of signaling NaN to quiet NaN by BrowserUk (Patriarch) on Jun 24, 2016 at 17:31 UTC
Re^5: unintentional conversion of signaling NaN to quiet NaN by syphilis (Archbishop) on Jun 25, 2016 at 11:00 UTC
Some notes below your chosen depth have not been shown here