in reply to Decimal numbers calculations done inconsistently: Odd behavior!
Your basic problem is that computers use representations of powers of 2 (even for fractional values), but not every fractional number can be represented accurately as such. For more information, a resource such as What Every Computer Scientist Should Know About Floating-Point Arithmetic or the link to which LanX response referred would be a good start.
As to a solution/fix, toolic's referred would be good starting places to look.
Hope that helps.
|
|---|