Re (tilly) 1: strange rounding behaviour

print int(0.5 + 0.695*100)/100;
[download]

prints 0.7 for me. Thereby showing that your round does not get the formatting correct which sprintf does, and still suffers from inconsistent rounding behaviour.

The reasons for this behaviour are basic to the fact of how floating point numbers work. They may not be readily predictable (exact details depend on your chip, OS, compiler, etc) but the underlying issue is that the internal representation of a floating point number is in base 2, while the visible one is in base 10. Therefore when you go to represent a floating point number, a nice round number base 10 turns into a repeating representation that is always stored with round-off error. What way you round in the boundary case depends on the interaction of that error and your operations.

Comment on Re (tilly) 1: strange rounding behaviour Download Code

Replies are listed 'Best First'.
Re: Re (tilly) 1: strange rounding behaviour by lhoward (Vicar) on May 18, 2001 at 20:35 UTC
Nice catch tilly. I missed that case. My opinion boils down to this: use string formatting routines for formatting strings (not rounding numbers), and use numeric tools for numeric operations (like rounding). Using a string operator (like sprintf) to do a numeric function (rounding) is just asking for "unexpected complications".	[reply]