in reply to Re^4: RFC: Large Floating Point Numbers - Rounding Errors
in thread RFC: Large Floating Point Numbers - Rounding Errors

YOU are wrong! Wrong! Wrong!

IEEE 64-bit doubles are perfectly capable of representing these values to sufficient accuracy to allow them to be rounded correctly.

See Re: RFC: Large Floating Point Numbers - Rounding Errors (Rounding mode)


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re^5: RFC: Large Floating Point Numbers - Rounding Errors

Replies are listed 'Best First'.
Re^6: RFC: Large Floating Point Numbers - Rounding Errors
by salva (Canon) on Sep 08, 2011 at 16:33 UTC
    No, they are not, this is a borderline case.

    Try the following C program and see how the periodic part of the base 2 representation of those rational numbers is truncated and how it converts back to base 10:

    #include <stdio.h> double n[] = { 0.000005, 0.000015, 0.000025, 0.000035, 0.000045, 0.000 +055, 0.000065 }; int main(int argc, char *argv[]) { int i; for (i = 0; i < sizeof(n)/sizeof(*n); i++) { printf("%40.30a => %40.30f\n", n[i], n[i]); } return 0; }
    On my amd64 linux box it outputs:
    0x1.4f8b588e368f100000000000000000p-18 => 0.0000050000000000 +00000409015270 0x1.f75104d551d6900000000000000000p-17 => 0.0000150000000000 +00000380012861 0x1.a36e2eb1c432d00000000000000000p-16 => 0.0000250000000000 +00001198043401 0x1.2599ed7c6fbd200000000000000000p-15 => 0.0000349999999999 +99996933876256 0x1.797cc39ffd60f00000000000000000p-15 => 0.0000450000000000 +00002834104479 0x1.cd5f99c38b04b00000000000000000p-15 => 0.0000550000000000 +00001958069124 0x1.10a137f38c54300000000000000000p-14 => 0.0000649999999999 +99994305770190
      ... this is a borderline case.

      Because YOU haven't set the rounding mode:

      #include <stdio.h> #include <float.h> int main( int argc, char **argv ) { double n; printf( "Setting mode CHOP; controlFP returned: %u\n", _controlfp( _RC_CHOP, _MCW_RC ) ); for( n = 0.000005; n < 0.0001; n += 0.00001 ) { printf( "%.15f %.5f\n", n, n ); } printf( "Setting mode UP; controlFP returned: %u\n", _controlfp( _RC_UP, _MCW_RC ) ); for( n = 0.000005; n < 0.0001; n += 0.00001 ) { printf( "%.15f %.5f\n", n, n ); } printf( "Setting mode DOWN; controlFP returned: %u\n", _controlfp( _RC_DOWN, _MCW_RC ) ); for( n = 0.000005; n < 0.0001; n += 0.00001 ) { printf( "%.15f %.5f\n", n, n ); } printf( "Setting mode NEAR; controlFP returned: %u\n", _controlfp( _RC_NEAR, _MCW_RC ) ); for( n = 0.000005; n < 0.0001; n += 0.00001 ) { printf( "%.15f %.5f\n", n, n ); } return 1; }
      C:\test>ieeeroundingmode.exe Setting mode CHOP; controlFP returned: 525087 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00003 0.000045000000000 0.00004 0.000055000000000 0.00005 0.000065000000000 0.00006 0.000075000000000 0.00007 0.000085000000000 0.00008 0.000095000000000 0.00009 Setting mode UP; controlFP returned: 524831 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00004 0.000045000000000 0.00005 0.000055000000000 0.00006 0.000065000000000 0.00007 0.000075000000000 0.00008 0.000085000000000 0.00009 0.000095000000000 0.00010 Setting mode DOWN; controlFP returned: 524575 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00003 0.000045000000000 0.00004 0.000055000000000 0.00005 0.000065000000000 0.00006 0.000075000000000 0.00007 0.000085000000000 0.00008 0.000095000000000 0.00009 Setting mode NEAR; controlFP returned: 524319 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00004 0.000045000000000 0.00005 0.000055000000000 0.00006 0.000065000000000 0.00007 0.000075000000000 0.00008 0.000085000000000 0.00009 0.000095000000000 0.00010

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        Sadly, Solaris doesn't implement _controlFP (nor does Inline exist in this environment). There is a Solaris method by which I can ask what the rounding method is on this system, just no way to set it.

        Like I said, something bigger is wrong here, and no, I shouldn't have to use a string manipulation program to deal with this, but ... I'm pretty stuck otherwise.
        You are cheating! can't you see it?