in reply to RFC: Large Floating Point Numbers - Rounding Errors

The problem is that your perl or C-library is setting (or leaving) the IEEE rounding mode set to RC_CHOP (truncate). Any of the other three modes -- UP, DOWN, OR NEAREST -- would fix this.

The following shows the output on my intel system for teh four modes of IEEE, FP operations:

#! perl -slw use strict; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'END_C', NAME => 'ieeroundmode', CLEAN_AFTER_BUILD +=> 0; unsigned int controlFP( unsigned int new, unsigned int mask ) { return _controlfp( new, mask ); } END_C use constant { RC_MASK => 0x00000300, RC_CHOP => 0x00000300, RC_UP => 0x00000200, RC_DOWN => 0x00000100, RC_NEAR => 0x00000000, }; print "Setting mode CHOP; controlFP returned: ", controlFP( RC_CHOP, R +C_MASK ); for( my $n = 0.000005; $n < 0.0001; $n += 0.00001 ) { printf "%.15f %.5f\n", $n, $n; } print "Setting mode UP; controlFP returned: ", controlFP( RC_UP, RC_MA +SK ); for( my $n = 0.000005; $n < 0.0001; $n += 0.00001 ) { printf "%.15f %.5f\n", $n, $n; } print "Setting mode DOWN; controlFP returned: ", controlFP( RC_DOWN, R +C_MASK ); for( my $n = 0.000005; $n < 0.0001; $n += 0.00001 ) { printf "%.15f %.5f\n", $n, $n; } print "Setting mode NEAR; controlFP returned: ", controlFP( RC_NEAR, R +C_MASK ); for( my $n = 0.000005; $n < 0.0001; $n += 0.00001 ) { printf "%.15f %.5f\n", $n, $n; } __END__ C:\test>ieeroundmode.pl Setting mode CHOP; controlFP returned: 525087 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00003 0.000045000000000 0.00004 0.000055000000000 0.00005 0.000065000000000 0.00006 0.000075000000000 0.00007 0.000085000000000 0.00008 0.000095000000000 0.00009 Setting mode UP; controlFP returned: 524831 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00004 0.000045000000000 0.00005 0.000055000000000 0.00006 0.000065000000000 0.00007 0.000075000000000 0.00008 0.000085000000000 0.00009 0.000095000000000 0.00010 Setting mode DOWN; controlFP returned: 524575 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00003 0.000045000000000 0.00004 0.000055000000000 0.00005 0.000065000000000 0.00006 0.000075000000000 0.00007 0.000085000000000 0.00008 0.000095000000000 0.00009 Setting mode NEAR; controlFP returned: 524319 0.000005000000000 0.00001 0.000015000000000 0.00002 0.000025000000000 0.00003 0.000035000000000 0.00004 0.000045000000000 0.00005 0.000055000000000 0.00006 0.000065000000000 0.00007 0.000075000000000 0.00008 0.000085000000000 0.00009 0.000095000000000 0.00010

Maybe you can set the rounding mode yourself in a similar way?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re: RFC: Large Floating Point Numbers - Rounding Errors (Rounding mode)
  • Download Code

Replies are listed 'Best First'.
Re^2: RFC: Large Floating Point Numbers - Rounding Errors (not Rounding mode)
by tye (Sage) on Sep 08, 2011 at 17:00 UTC

    Nice try. But...

    No. It isn't.

    YOU are wrong! Wrong! Wrong!

    It is just a consequence of switching between base 10 and 2:

    #!/usr/bin/perl -wl use strict; for my $n ( 0.000005, 0.000015, 0.000025, 0.000035, 0.000045, 0.000055, 0.000065, 0.000075, 0.000085, 0.000095, ) { printf "%.23f\n", $n; } __END__ 0.00000500000000000000041 0.00001500000000000000038 0.00002500000000000000120 0.00003499999999999999693 0.00004500000000000000283 0.00005500000000000000196 0.00006499999999999999431 0.00007499999999999999343 0.00008500000000000000611 0.00009500000000000000523

    Your assumption that 0.000035 is exactly the same as 0.000005 + 3*0.000010 was not correct.

    Update: Please show the results for explicitly enumerated values and show more precision on the first value. Or I'll to do that myself a bit later when I will probably again have convenient access to a system with Inline::C installed on it and with _controlfp().

    - tye        

Re^2: RFC: Large Floating Point Numbers - Rounding Errors (absolutely not rounding mode)
by tye (Sage) on Sep 11, 2011 at 07:21 UTC

    Well, you got rather quiet on the subject but I wasn't convinced that there might not be some truth to the "rounding mode" stuff. But it turns out that there was no truth to it at all for this case.

    The rounding mode is already at NEAR and the rounding mode makes no difference in the behavior of sprintf()'s rounding. The reason you got results more to your liking is purely due to computing 0.000035 using repeated additions rather than using the constant 0.000035 directly.

    Here is how I modified your test code to prove this:

    And here is summarized output, collapsing identical sections and highlighting the differences between the different parts:

    Default / NEAR (compare computed value with cons +tant) 0.0000050000000000000004 0.00001 =0.0000050000000000000004= =0.000 +01= 0.0000150000000000000000 0.00002 <0.0000150000000000000020> =0.000 +02= 0.0000250000000000000010 0.00003 <0.0000250000000000000050> =0.000 +03= 0.0000349999999999999970 0.00003 <0.0000350000000000000040> <0.000 +04> 0.0000450000000000000030 0.00005 =0.0000450000000000000030= =0.000 +05= 0.0000550000000000000020 0.00006 =0.0000550000000000000020= =0.000 +06= 0.0000649999999999999940 0.00006 <0.0000650000000000000080> <0.000 +07> 0.0000749999999999999930 0.00007 <0.0000750000000000000070> <0.000 +08> 0.0000850000000000000060 0.00009 =0.0000850000000000000060= =0.000 +09= 0.0000950000000000000050 0.00010 =0.0000950000000000000050= =0.000 +10= CHOP / DOWN (compare value to Default / NEAR case) >0.0000049999999999999996< >0.00000< >0.0000049999999999999996< >0.000 +00< >0.0000149999999999999990< >0.00001< >0.0000150000000000000000< =0.000 +02= >0.0000249999999999999980< >0.00002< >0.0000250000000000000010< =0.000 +03= =0.0000349999999999999970= =0.00003= >0.0000349999999999999970< >0.000 +03< >0.0000449999999999999960< >0.00004< >0.0000449999999999999960< >0.000 +04< >0.0000549999999999999950< >0.00005< >0.0000549999999999999950< >0.000 +05< >0.0000649999999999999940< >0.00006< >0.0000649999999999999940< >0.000 +06< =0.0000749999999999999930= >0.00007< >0.0000749999999999999930< >0.000 +07< >0.0000849999999999999930< >0.00008< >0.0000849999999999999930< >0.000 +08< >0.0000949999999999999920< >0.00009< >0.0000949999999999999920< >0.000 +09< UP (compare value to Default / NEAR case) =0.0000050000000000000004= =0.00001= =0.0000050000000000000004= =0.000 +01= =0.0000150000000000000000= =0.00002= =0.0000150000000000000020= =0.000 +02= =0.0000250000000000000010= =0.00003= =0.0000250000000000000050> =0.000 +03= <0.0000350000000000000040> <0.00004> <0.0000350000000000000100> =0.000 +04= =0.0000450000000000000030= =0.00005= <0.0000450000000000000160> =0.000 +05= =0.0000550000000000000020= =0.00006= <0.0000550000000000000220> =0.000 +06= <0.0000650000000000000080> <0.00007> <0.0000650000000000000350> =0.000 +07= <0.0000750000000000000070> <0.00008> <0.0000750000000000000480> =0.000 +08= =0.0000850000000000000060= =0.00009= <0.0000850000000000000600> =0.000 +09= =0.0000950000000000000050= =0.00010= <0.0000950000000000000730> =0.000 +10=

    Setting rounding mode to NEAR gives the exact same results as I get before changing the rounding mode. Setting it to UP gives the desired results (for my code) not because it changes how sprintf rounds but because it changes how Perl converts the string '0.000035' into a floating point value, ensuring that the result is always slightly higher than (or equal to) the value that the string represents. But using UP also introduces growing inaccuracy in the additions.

    The "better" results in your NEAR case were only the luck of 0.000005 and 0.000010 converting to floating point values slightly larger than the numbers those strings represent.

    Also note that your results would be slightly different than mine for the DOWN / CHOP cases, since you put the constant 0.000005 in as a number so that it was interpreted at compile time (and thus using the NEAR rounding mode) rather than the way I used strings to force run-time interpretation of the constants using the different round modes (except for 0.000010).

    - tye        

      Well, you got rather quiet on the subject

      When the discussion turns from attempting to solve the OPs problem, to either a pissing contest or a witch hunt, participating further is of little purpose.

      The reason you got results more to your liking is purely due to computing 0.000035 using repeated additions rather than using the constant 0.000035 directly.

      So, what you are saying is, my "mistake" (or "cheat") was that I used numbers for (shock horror!) computation.

      Obviously, the affect of the float point processors rounding mode will only have a significant affect if you actually perform some -- ta-da! -- floating point calculations.

      I guess you could term sprintf "%.5f", '0.000035'; a calculation, but given that '0.00004'; is just easier and clearer, you have to wonder at the purpose.

      Rounding (and the effects of rounding mode upon it) only becomes meaningful when outputting the results of a computation. And it isn't just addition, it is all operations.

      There are an infinite number of calculations, that in an ideal world would produce exactly 0.000035. Some of them will round out one way, and some the other:

      printf "%.23f : %.5f\n", $_, $_ for 0.000005 * 7;; 0.00003500000000000000400 : 0.00004 printf "%.23f : %.5f\n", $_, $_ for 0.000007 * 5;; 0.00003499999999999999700 : 0.00003

      The truncation of a 6 sig.fig constant to a 5 sig.fig. string is the least interesting -- read: pointless -- of them all.

      And using the appropriate rounding mode, not just for the final calculation, but also all intermediate terms becomes very important as the number and complexity of the operations involved increases.

      I stick by my statement that a 64-bit IEEE 754 floating point value is perfectly capable of storing 0.000035 to sufficient accuracy that it can be rounded correctly, for any mathematically significant purpose.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.