in reply to 64-bit build fails numeric tests
1/10 is a periodic numbers in binary (like 1/3 is in decimal). It can't be represented exactly as a float. Normally, the least significant bits are rounded off to give the appearance of being able to store those numbers. You're not getting that rounding for some reason.
I find it odd that a 64-bit build uses double-precision numbers, but I don't know if that's unusual or wrong. I'm not experienced with those.
If you don't get a solution here, I suggest that you file a bug report. It'll put you in touch with people better equipped to solve this problem.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: 64-bit build fails numeric tests
by syphilis (Archbishop) on Feb 02, 2010 at 13:45 UTC | |
by eshafto (Novice) on Feb 02, 2010 at 16:23 UTC |