> If the math never used anything but all integer values, then this output precision issue doesn't appear?
Yes!
(well you could still knock at integer limits and need to resort to bigint, but that's another problem)
This phenomenon already caused much sorrow in financial calculations where cents where inaccurate, and fiscal authorities can be very nasty (or nazi) about inaccurate cents.
The rule of thumb is, if you want n decimal points accuracy, do a n points left shift before starting and an n point right shift of the results.
Roughly speaking: If you need accurate cents, calculate in cents and only convert the result into dollars. (I'd do 1/100th cents to be sure)
Like this all relevant problems between decimal and binary calculations will stay beyond error margin.
Nota Bene: This won't solve the problem of error propagation if you are doing loads of calculations, but financial businesses normally define explicit rounding rules to normalize this.
Cheers Rolf
(addicted to the Perl Programming Language and ☆☆☆☆ :)
Je suis Charlie!
In reply to Re^5: Integers sometimes turn into Reals after substraction (printed accuracy and float subtraction)
by LanX
in thread Integers sometimes turn into Reals after substraction
by rduke15
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |
