Re^2: Decimal precision issue: Windows vs. Unix

IEEE 754 requires correct rounding for numbers of this sort of size.

IEEE 754 has four rounding modes(bottom of the page.

Nearest.
Rounds to the nearest value, except for 5 which is equidistant, in which case it rounds to the nearest even digit.
Up
Next higher
Down.
Next lower.
Chop.
Truncate.

All are valid.

The MS CRT provides _control87 and varients to allow the user to choose which mode they require. With the default being nearest.

In the case of -95.24436542871095 (the fullest precision available from a 8-byte float), with the last digit (5) being equidistant from ...09 and ...10, it rounds to the even value. Per the spec.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Comment on Re^2: Decimal precision issue: Windows vs. Unix

Replies are listed 'Best First'.
Re^3: Decimal precision issue: Windows vs. Unix by gone2015 (Deacon) on Jan 11, 2009 at 00:25 UTC
As you say, by default one would expect round to nearest. I accept that I did not make that explicit... I apologise to anyone who was confused by my failure to be clear that I was not addressing the possible use of any of the Directed Roundings in this case. On the topic of rounding, IEEE 754 says: 4. Rounding Rounding takes a number regarded as infinitely precise and, if necessary, modifies it to fit in the destination's format... Section 5.6 of the standard specifies that "conversions shall be correctly rounded as specified in Section 4", for a range of numbers which includes those under discussion. So, in the case in point, the rounding decision for the binary decimal conversion to 15 significant decimal digits should start from approximately -95.244365428710949572632671333849 if it is to be 'correctly rounded'. Starting from the already rounded 95.24436542871095 gives the wrong result, as previously discussed. For more on the standard there's Supplemental Readings for IEEE 754 / 854. Jerome Coonen's "Contributions to a Proposed Standard for Binary Floating-Point Arithmetic" has a complete chapter on "Accurate yet Economical Binary-Decimal Conversions", which I can recommend.	[reply]

Replies are listed 'Best First'.

Re^3: Decimal precision issue: Windows vs. Unix
by gone2015 (Deacon) on Jan 11, 2009 at 00:25 UTC

As you say, by default one would expect round to nearest. I accept that I did not make that explicit... I apologise to anyone who was confused by my failure to be clear that I was not addressing the possible use of any of the Directed Roundings in this case.

On the topic of rounding, IEEE 754 says:

4. Rounding

Rounding takes a number regarded as infinitely precise and, if necessary, modifies it to fit in the destination's format...

Section 5.6 of the standard specifies that "conversions shall be correctly rounded as specified in Section 4", for a range of numbers which includes those under discussion.

So, in the case in point, the rounding decision for the binary decimal conversion to 15 significant decimal digits should start from approximately -95.244365428710949572632671333849 if it is to be 'correctly rounded'. Starting from the already rounded 95.24436542871095 gives the wrong result, as previously discussed.

For more on the standard there's Supplemental Readings for IEEE 754 / 854. Jerome Coonen's "Contributions to a Proposed Standard for Binary Floating-Point Arithmetic" has a complete chapter on "Accurate yet Economical Binary-Decimal Conversions", which I can recommend.

[reply]