in reply to Decimal precision issue: Windows vs. Unix

To look at this more closely I tried:

my $a = -95.3009707988281 ; show($a) ; my $b = -95.1877600585938 ; show($b) ; my $x = ($a + $b) / 2 ; show($x) ; printf "%20.16f\n", $x ; sub show { my ($f) = @_ ; printf "%-18s 0x%04X_%04X_%04X_%04X\n", $f, unpack("n4", pack("d>", +$f)) ; } ;
this gave:
Windows, perl 5.10.0, 32-bit               Linux, perl 5.10.0, 64-bit
-95.3009707988281  0xC057_D343_1B06_8122   -95.3009707988281  0xC057_D343_1B06_8122
-95.1877600585938  0xC057_CC04_42C3_C9F2   -95.1877600585938  0xC057_CC04_42C3_C9F2
-95.244365428711   0xC057_CFA3_AEE5_258A   -95.2443654287109  0xC057_CFA3_AEE5_258A
-95.2443654287109500                       -95.2443654287109496
which shows that the decimal to binary conversion is giving the same result, as is the arithmetic. What is different is the binary to decimal conversion. It would appear that Perl is stringifying to 15 significant decimal digits, discarding any trailing zeros. Other experiments suggest that the library under Windows is returning 17 significant decimal digits, but the library under Linux is returning rather more -- giving a different result when rounded to 15 decimal digits.

Binary/Decimal conversion is a whole lot trickier than it looks. Producing no more than 17 decimal digits is not unreasonable for IEEE 754 double precision floats. On the face of it, however, what we have here is a double rounding under Windows, which I think is incorrect.

Nevertheless, it is "ambitious" to expect any two floating point values to be exactly equal !

Replies are listed 'Best First'.
Re^2: Decimal precision issue: Windows vs. Unix
by ikegami (Patriarch) on Jan 10, 2009 at 01:12 UTC
    17 digits of precision is the max for doubles. Are you aware that your linux perl is using long doubles?
      Are you aware that your linux perl is using long doubles?

      I don't think it is. For the expression presented in the original post in this thread, on linux (without long double support) I get the same -95.2443654287109, but on the same machine (and OS), when perl has been built with -Duselongdouble, the output is -95.24436542871095.

      Of course, the output of perl -V:archname would tell us for sure.

      Cheers,
      Rob

      No, I'm not aware of it using long doubles. perl -V gives:

      Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
        Platform:
          osname=linux, osvers=2.6.18-92.1.10.el5, archname=x86_64-linux-thread-multi
      ....
          intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
          d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
          ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
          alignbytes=8, prototype=define
      ....
      
      which suggests to me that it's using ordinary 8 byte doubles.

        longdblsize=16