in reply to Decimal precision issue: Windows vs. Unix

Not to detract from the general point that you should never rely on the exact value of a floating point number...

...I am increasingly of the opinion that this demonstates a fault in the Windows library.

When I try it on my Windows XP 32-bit perl 5.10.0, and on my Linux 64-bit perl 5.10.0, the result of the sum is the same, to whit an IEEE 754 double 0xC057_CFA3_AEE5_258A, which is approximately -95.244365428710949572632671333849

It appears that when stringifying floating value in IEEE 754 double form, Perl is working to 15 significant decimal digits. This is the usual value -- chosen because this ensures that conversion from decimal to binary and back again gives the original decimal value. Under Windows the perl I have is using sprintf(b, "%.*g", 15, x), where under Linux it is using gcvt(x, 15, b) -- at least that is what use Config tells me d_Gconvert is set to.

Anyway, as you know, the results are:

  -95.2443654287109                  -- Linux
  -95.244365428710949572632671333849 ~ true value
  -95.244365428711                   -- Windows

[download]
which demonstrates that the Linux value is 'correctly rounded' to 15 significant decimal digits, while the Windows value... isn't. IEEE 754 requires correct rounding for numbers of this sort of size.

One of the tricky things about binary to decimal conversion is: to do it right you have to delay a rounding step until after you've generated the decimal value, and then round to the required number of decimals. It is reasonable to limit the conversion of IEEE 754 doubles to 17 significant decimal digits -- because that is sufficient for binary to decimal and back to binary to give the original value, for reasonable size values.

You can see that when rounded to 17 significant digits the value is -95.244365428710950, rounding that to 15 digits gives -95.244365428711. I observe that sprintf('%24.20f', ...) under Windows gives zeros after the 17th significant digit...

Replies are listed 'Best First'.
Re^2: Decimal precision issue: Windows vs. Unix
by BrowserUk (Patriarch) on Jan 10, 2009 at 21:04 UTC
    IEEE 754 requires correct rounding for numbers of this sort of size.

    IEEE 754 has four rounding modes(bottom of the page.

    1. Nearest.

      Rounds to the nearest value, except for 5 which is equidistant, in which case it rounds to the nearest even digit.

    2. Up

      Next higher

    3. Down.

      Next lower.

    4. Chop.

      Truncate.

    All are valid.

    The MS CRT provides _control87 and varients to allow the user to choose which mode they require. With the default being nearest.

    In the case of -95.24436542871095 (the fullest precision available from a 8-byte float), with the last digit (5) being equidistant from ...09 and ...10, it rounds to the even value. Per the spec.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

      As you say, by default one would expect round to nearest. I accept that I did not make that explicit... I apologise to anyone who was confused by my failure to be clear that I was not addressing the possible use of any of the Directed Roundings in this case.

      On the topic of rounding, IEEE 754 says:

      4. Rounding

      Rounding takes a number regarded as infinitely precise and, if necessary, modifies it to fit in the destination's format...

      Section 5.6 of the standard specifies that "conversions shall be correctly rounded as specified in Section 4", for a range of numbers which includes those under discussion.

      So, in the case in point, the rounding decision for the binary decimal conversion to 15 significant decimal digits should start from approximately -95.244365428710949572632671333849 if it is to be 'correctly rounded'. Starting from the already rounded 95.24436542871095 gives the wrong result, as previously discussed.

      For more on the standard there's Supplemental Readings for IEEE 754 / 854. Jerome Coonen's "Contributions to a Proposed Standard for Binary Floating-Point Arithmetic" has a complete chapter on "Accurate yet Economical Binary-Decimal Conversions", which I can recommend.

[reply]

In Section Seekers of Perl Wisdom