Re: Decimal precision issue: Windows vs. Unix

If they aren't exactly the same machine, I'd say that's likely a hardware issue: You're dealing with the limits of the precision of the machine, and how the processor rounds numbers when it can't give the exact value.

If you need to work with numbers like this, take good look at Math::BigInt.

Comment on Re: Decimal precision issue: Windows vs. Unix

Replies are listed 'Best First'.
Re^2: Decimal precision issue: Windows vs. Unix by ikegami (Patriarch) on Jan 09, 2009 at 21:54 UTC
I imaging that differences in the C library could affect the result as well. In fact, I'd say it's likely here since both numbers are the same, just with a different number of digits being printed. Update: Confirmed `$x = ( -95.3009707988281 + -95.1877600585938 ) / 2; $h = uc unpack 'H*', reverse pack 'd', $x; printf("%s\n%.16e\n%s\n", $h, $x, $x);` [download] gives `# MS cl, Windows, x86 C057CFA3AEE5258A -9.5244365428710950e+001 -95.244365428711 # gcc, linux, x86 C057CFA3AEE5258A -9.5244365428710950e+01 -95.2443654287109` [download] The number is exactly the same, it's just the conversion to text by the C library that outputs a different number of digits.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: Decimal precision issue: Windows vs. Unix
by ikegami (Patriarch) on Jan 09, 2009 at 21:54 UTC

Update: Confirmed

$x = ( -95.3009707988281 + -95.1877600585938 ) / 2;
$h = uc unpack 'H*', reverse pack 'd', $x;
printf("%s\n%.16e\n%s\n", $h, $x, $x);
[download]

gives

# MS cl, Windows, x86
C057CFA3AEE5258A
-9.5244365428710950e+001
-95.244365428711

# gcc, linux, x86
C057CFA3AEE5258A
-9.5244365428710950e+01
-95.2443654287109
[download]

The number is exactly the same, it's just the conversion to text by the C library that outputs a different number of digits.

[reply]
[d/l]
[select]