in reply to Re^2: Need more precision.
in thread Need more precision.
Got that, but Fixed-point arithmetic "is essentially an integer that is scaled by an implicit specific factor determined by the type. For example, the value 1.23 can be represented as 1230 in a fixed-point data type with scaling factor of 1/1000". This should work since you said your min/max is -4.0 to 4.0, which could be represented in a 128-bit integer as, for example, 0 to 80000000000000000000000000000000000000.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^4: Need more precision.
by BrowserUk (Patriarch) on Jun 09, 2015 at 22:37 UTC | |
by Anonymous Monk on Jun 09, 2015 at 22:53 UTC | |
by BrowserUk (Patriarch) on Jun 09, 2015 at 23:05 UTC | |
by Anonymous Monk on Jun 10, 2015 at 00:05 UTC | |
by BrowserUk (Patriarch) on Jun 10, 2015 at 00:28 UTC | |
|