in reply to Behaviour of int() unexpected
What Every Computer Scientist Should Know About Floating-Point Arithmetic
Also, if your perl is new enough, the %a or %A sprintf conversion will show the hex version of the internal floating-point binary representation used:
linux shell: perl -f -e 'printf qq(%A\n), $_ for 8.95, 8.95*100, 895' windows cmd.exe: perl -f -e "printf qq(%A\n), $_ for 8.95, 8.95*100, 895"
Mine shows:
0X1.1E66666666666P+3 0X1.BF7FFFFFFFFFFP+9 0X1.BF8P+9
update: the anonymous post slightly beat me to the punch, though showing a single-precision float value, rather than the double-precision float that is more likely for your perl to be using. In this instance, they both are slightly less than 8.95 originally, so both round to 894 after your multiplication; but in some cases, the single and double-precision values fall on different sides of the rouding boundary, so rounding a float and rounding a double give different values. Everyone who is ever going to program a floating-point application should watch Superman III. (Okay, that's showing my age; slightly more recent is Office Space.)
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^2: Behaviour of int() unexpected
by ikegami (Patriarch) on Mar 11, 2025 at 00:31 UTC | |
Re^2: Behaviour of int() unexpected
by harangzsolt33 (Deacon) on Mar 11, 2025 at 00:33 UTC | |
by ikegami (Patriarch) on Mar 11, 2025 at 00:41 UTC | |
by harangzsolt33 (Deacon) on Mar 11, 2025 at 02:41 UTC | |
by LanX (Saint) on Mar 11, 2025 at 03:59 UTC | |
by syphilis (Archbishop) on Mar 11, 2025 at 10:06 UTC | |
by LanX (Saint) on Mar 13, 2025 at 03:30 UTC | |
by ikegami (Patriarch) on Mar 11, 2025 at 03:25 UTC | |
by syphilis (Archbishop) on Mar 11, 2025 at 03:30 UTC | |
by LanX (Saint) on Mar 11, 2025 at 03:48 UTC | |
by cavac (Prior) on Mar 13, 2025 at 15:27 UTC | |
by LanX (Saint) on Mar 13, 2025 at 20:25 UTC | |
by cavac (Prior) on Mar 14, 2025 at 07:36 UTC |