Re^2: Decimal numbers calculations done inconsistently: Odd behavior!

This works well if you expect the floating point number to be near an integer. If you expect 3.5 to round to 4, it will fail if your 3.5 is really 3.49999999...

Bill

Comment on Re^2: Decimal numbers calculations done inconsistently: Odd behavior!

Replies are listed 'Best First'.
Re^3: Decimal numbers calculations done inconsistently: Odd behavior! by Laurent_R (Canon) on Mar 19, 2015 at 22:14 UTC
That may be slightly more complicated than that. IEEE Standard rounding rules are somewhat unexpected to many people. This is what true rounding is supposed to be according to IEEE: `$ perl -e 'my $c = 0.5; $c +=1 and printf "%f -> %0.0f\n", $c, $c for +1..10;' 1.500000 -> 2 2.500000 -> 2 3.500000 -> 4 4.500000 -> 4 5.500000 -> 6 6.500000 -> 6 7.500000 -> 8 8.500000 -> 8 9.500000 -> 10 10.500000 -> 10` [download] Round up if the previous digit is odd and round down if it is even. Je suis Charlie.	[reply] [d/l]

Replies are listed 'Best First'.

Re^3: Decimal numbers calculations done inconsistently: Odd behavior!
by Laurent_R (Canon) on Mar 19, 2015 at 22:14 UTC

true rounding

$ perl -e 'my $c = 0.5; $c +=1 and printf "%f -> %0.0f\n", $c, $c for 
+1..10;'
1.500000 -> 2
2.500000 -> 2
3.500000 -> 4
4.500000 -> 4
5.500000 -> 6
6.500000 -> 6
7.500000 -> 8
8.500000 -> 8
9.500000 -> 10
10.500000 -> 10
[download]

Je suis Charlie.

[reply]
[d/l]