in reply to Re: OT: How much float precision needed for operation?
in thread OT: How much float precision needed for operation?
To ensure an error of no more than plus/minus 0.1 we'd need ~7 digits to the right of the decimal point for y=10 and ~53 digits for y=100Is this correct?
-Scott
Update: trammel, thank you very much for your excellent examples above.
So, for others who want to extend this to other functions:
If we take the derivative of the function we're working with, multiplied by a delta (difference), and know what our tolerance for error is, we can solve for the maximum delta that will be within our tolerance for error.In math symbols, the process is:
- given function F
- find the derative -> dF
- maxdelta = errortolerance/dF
|
---|