in reply to Re^4: number comparison with a twist
in thread number comparison with a twist
I understand that the internal representation is (probably) floating point. However that is still an assumption of what the internal representation "decimal string" actually is. My point is that using the eq operator targets "decimal string" which has a lower precision (for DWIM reasons?), so you are absolutely right that that method smoothes the rounding errors from floats away
printf "%.20f",$varThis targeted floating point with high precision.
This is what I (presumably) hope to target with the eq:
printf "%.11f", $num2*100 ; # 1990.00000000000I'm not sure if $.11f now shows the exact correct number of decimals (which should be 15 I believe), but as far as I can see this solves OP's problem and that is why I suggested it. Just using eq is in my opinion fine for this particular use case. For the rest floating point is always a pain in the... well, you know
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^6: number comparison with a twist
by LanX (Saint) on Mar 02, 2020 at 18:22 UTC | |
by Veltro (Hermit) on Mar 02, 2020 at 21:18 UTC | |
by LanX (Saint) on Mar 03, 2020 at 00:55 UTC |