in reply to Re^2: RFC: Large Floating Point Numbers - Rounding Errors
in thread RFC: Large Floating Point Numbers - Rounding Errors

Very strange. I'd almost say that you'd discovered a bug in perl or the underlying C-libraries that was causing the double precision (64-bit) NV value to be transitioned through a single precision (32-bit) float whilst being formatted.

If the values were always being manipulated as doubles, this shouldn't happen.

It could be a that the IEEE rounding mode isn't being set (correctly?).

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re^3: RFC: Large Floating Point Numbers - Rounding Errors

Replies are listed 'Best First'.
Re^4: RFC: Large Floating Point Numbers - Rounding Errors
by salva (Canon) on Sep 08, 2011 at 16:10 UTC
    This is not a bug but the result of converting numbers between base 10 and 2.

    The problem is that 0.000005, 0.000015, 0.000025, 0.000035, 0.000045, 0.000055, 0.000065, ... can not be represented precisely in base 2: 

    printf("%.5f %.30f\n", $_, $_) for (0.000005, 0.000015, 0.000025, 0.00
+0035, 0.000045, 0.000055, 0.000065)

    [download]
[reply]
[d/l]
      the result of converting numbers between base 10 and 2.

      No. It isn't.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

      YOU are wrong! Wrong! Wrong!

      IEEE 64-bit doubles are perfectly capable of representing these values to sufficient accuracy to allow them to be rounded correctly.

      See Re: RFC: Large Floating Point Numbers - Rounding Errors (Rounding mode)

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        No, they are not, this is a borderline case.

        Try the following C program and see how the periodic part of the base 2 representation of those rational numbers is truncated and how it converts back to base 10: 

        #include <stdio.h>

double n[] = { 0.000005, 0.000015, 0.000025, 0.000035, 0.000045, 0.000
+055, 0.000065 };

int main(int argc, char *argv[]) {
    int i;
    for (i = 0; i < sizeof(n)/sizeof(*n); i++) {
        printf("%40.30a => %40.30f\n", n[i], n[i]);
    }
    return 0;
}

        [download]
        On my amd64 linux box it outputs: 
          0x1.4f8b588e368f100000000000000000p-18 =>         0.0000050000000000
+00000409015270
  0x1.f75104d551d6900000000000000000p-17 =>         0.0000150000000000
+00000380012861
  0x1.a36e2eb1c432d00000000000000000p-16 =>         0.0000250000000000
+00001198043401
  0x1.2599ed7c6fbd200000000000000000p-15 =>         0.0000349999999999
+99996933876256
  0x1.797cc39ffd60f00000000000000000p-15 =>         0.0000450000000000
+00002834104479
  0x1.cd5f99c38b04b00000000000000000p-15 =>         0.0000550000000000
+00001958069124
  0x1.10a137f38c54300000000000000000p-14 =>         0.0000649999999999
+99994305770190

        [download]
[reply]
[d/l]
[select]

In Section Meditations