in reply to Re^4: Need more precision.
in thread Need more precision.
0.00000000000000002025 which becomes 0.0000000000000000 with doubles.
Thinking on this more, I wonder if you are really having string-ification problem. Digits could be getting lost in that process. The value in normalized notation is 2.025 * 10^-17, which is well within the capabilities of doubles, which use double-precision floating-point format
Update: Fixed a typo.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^6: Need more precision.
by BrowserUk (Patriarch) on Jun 14, 2015 at 13:18 UTC | |
by RonW (Parson) on Jun 15, 2015 at 18:31 UTC |