in reply to Re: RFC: Large Floating Point Numbers - Rounding Errors (Rounding mode)
in thread RFC: Large Floating Point Numbers - Rounding Errors

Well, you got rather quiet on the subject but I wasn't convinced that there might not be some truth to the "rounding mode" stuff. But it turns out that there was no truth to it at all for this case.

The rounding mode is already at NEAR and the rounding mode makes no difference in the behavior of sprintf()'s rounding. The reason you got results more to your liking is purely due to computing 0.000035 using repeated additions rather than using the constant 0.000035 directly.

Here is how I modified your test code to prove this:

#!/usr/bin/perl -lw use strict; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'END_C', NAME => 'ieeroundmode', CLEAN_AFTER_BUILD +=> 0; unsigned int controlFP( unsigned int new, unsigned int mask ) { return _controlfp( new, mask ); } END_C use constant { RC_MASK => 0x00000300, RC_CHOP => 0x00000300, RC_UP => 0x00000200, RC_DOWN => 0x00000100, RC_NEAR => 0x00000000, }; my @n; my $m; @n = qw< 0.000005 0.000015 0.000025 0.000035 0.000045 0.000055 0.000065 0.000075 0.000085 0.000095 >; $m= $n[0]; for my $n ( @n ) { printf " %.22f %.5f %.22f %.5f\n", $n, $n, $m, $m; $m += 0.00001; } print "Setting mode CHOP; controlFP returned: ", controlFP( RC_CHOP, R +C_MASK ); @n = qw< 0.000005 0.000015 0.000025 0.000035 0.000045 0.000055 0.000065 0.000075 0.000085 0.000095 >; $m= $n[0]; for my $n ( @n ) { printf " %.22f %.5f %.22f %.5f\n", $n, $n, $m, $m; $m += 0.00001; } print "Setting mode UP; controlFP returned: ", controlFP( RC_UP, RC_MA +SK ); @n = qw< 0.000005 0.000015 0.000025 0.000035 0.000045 0.000055 0.000065 0.000075 0.000085 0.000095 >; $m= $n[0]; for my $n ( @n ) { printf " %.22f %.5f %.22f %.5f\n", $n, $n, $m, $m; $m += 0.00001; } print "Setting mode DOWN; controlFP returned: ", controlFP( RC_DOWN, R +C_MASK ); @n = qw< 0.000005 0.000015 0.000025 0.000035 0.000045 0.000055 0.000065 0.000075 0.000085 0.000095 >; $m= $n[0]; for my $n ( @n ) { printf " %.22f %.5f %.22f %.5f\n", $n, $n, $m, $m; $m += 0.00001; } print "Setting mode NEAR; controlFP returned: ", controlFP( RC_NEAR, R +C_MASK ); @n = qw< 0.000005 0.000015 0.000025 0.000035 0.000045 0.000055 0.000065 0.000075 0.000085 0.000095 >; $m= $n[0]; for my $n ( @n ) { printf " %.22f %.5f %.22f %.5f\n", $n, $n, $m, $m; $m += 0.00001; }

And here is summarized output, collapsing identical sections and highlighting the differences between the different parts:

Default / NEAR (compare computed value with cons +tant) 0.0000050000000000000004 0.00001 =0.0000050000000000000004= =0.000 +01= 0.0000150000000000000000 0.00002 <0.0000150000000000000020> =0.000 +02= 0.0000250000000000000010 0.00003 <0.0000250000000000000050> =0.000 +03= 0.0000349999999999999970 0.00003 <0.0000350000000000000040> <0.000 +04> 0.0000450000000000000030 0.00005 =0.0000450000000000000030= =0.000 +05= 0.0000550000000000000020 0.00006 =0.0000550000000000000020= =0.000 +06= 0.0000649999999999999940 0.00006 <0.0000650000000000000080> <0.000 +07> 0.0000749999999999999930 0.00007 <0.0000750000000000000070> <0.000 +08> 0.0000850000000000000060 0.00009 =0.0000850000000000000060= =0.000 +09= 0.0000950000000000000050 0.00010 =0.0000950000000000000050= =0.000 +10= CHOP / DOWN (compare value to Default / NEAR case) >0.0000049999999999999996< >0.00000< >0.0000049999999999999996< >0.000 +00< >0.0000149999999999999990< >0.00001< >0.0000150000000000000000< =0.000 +02= >0.0000249999999999999980< >0.00002< >0.0000250000000000000010< =0.000 +03= =0.0000349999999999999970= =0.00003= >0.0000349999999999999970< >0.000 +03< >0.0000449999999999999960< >0.00004< >0.0000449999999999999960< >0.000 +04< >0.0000549999999999999950< >0.00005< >0.0000549999999999999950< >0.000 +05< >0.0000649999999999999940< >0.00006< >0.0000649999999999999940< >0.000 +06< =0.0000749999999999999930= >0.00007< >0.0000749999999999999930< >0.000 +07< >0.0000849999999999999930< >0.00008< >0.0000849999999999999930< >0.000 +08< >0.0000949999999999999920< >0.00009< >0.0000949999999999999920< >0.000 +09< UP (compare value to Default / NEAR case) =0.0000050000000000000004= =0.00001= =0.0000050000000000000004= =0.000 +01= =0.0000150000000000000000= =0.00002= =0.0000150000000000000020= =0.000 +02= =0.0000250000000000000010= =0.00003= =0.0000250000000000000050> =0.000 +03= <0.0000350000000000000040> <0.00004> <0.0000350000000000000100> =0.000 +04= =0.0000450000000000000030= =0.00005= <0.0000450000000000000160> =0.000 +05= =0.0000550000000000000020= =0.00006= <0.0000550000000000000220> =0.000 +06= <0.0000650000000000000080> <0.00007> <0.0000650000000000000350> =0.000 +07= <0.0000750000000000000070> <0.00008> <0.0000750000000000000480> =0.000 +08= =0.0000850000000000000060= =0.00009= <0.0000850000000000000600> =0.000 +09= =0.0000950000000000000050= =0.00010= <0.0000950000000000000730> =0.000 +10=

Setting rounding mode to NEAR gives the exact same results as I get before changing the rounding mode. Setting it to UP gives the desired results (for my code) not because it changes how sprintf rounds but because it changes how Perl converts the string '0.000035' into a floating point value, ensuring that the result is always slightly higher than (or equal to) the value that the string represents. But using UP also introduces growing inaccuracy in the additions.

The "better" results in your NEAR case were only the luck of 0.000005 and 0.000010 converting to floating point values slightly larger than the numbers those strings represent.

Also note that your results would be slightly different than mine for the DOWN / CHOP cases, since you put the constant 0.000005 in as a number so that it was interpreted at compile time (and thus using the NEAR rounding mode) rather than the way I used strings to force run-time interpretation of the constants using the different round modes (except for 0.000010).

- tye        

  • Comment on Re^2: RFC: Large Floating Point Numbers - Rounding Errors (absolutely not rounding mode)
  • Select or Download Code

Replies are listed 'Best First'.
Re^3: RFC: Large Floating Point Numbers - Rounding Errors (truncating constants is uninteresting)
by BrowserUk (Patriarch) on Sep 11, 2011 at 08:10 UTC
    Well, you got rather quiet on the subject

    When the discussion turns from attempting to solve the OPs problem, to either a pissing contest or a witch hunt, participating further is of little purpose.

    The reason you got results more to your liking is purely due to computing 0.000035 using repeated additions rather than using the constant 0.000035 directly.

    So, what you are saying is, my "mistake" (or "cheat") was that I used numbers for (shock horror!) computation.

    Obviously, the affect of the float point processors rounding mode will only have a significant affect if you actually perform some -- ta-da! -- floating point calculations.

    I guess you could term sprintf "%.5f", '0.000035'; a calculation, but given that '0.00004'; is just easier and clearer, you have to wonder at the purpose.

    Rounding (and the effects of rounding mode upon it) only becomes meaningful when outputting the results of a computation. And it isn't just addition, it is all operations.

    There are an infinite number of calculations, that in an ideal world would produce exactly 0.000035. Some of them will round out one way, and some the other:

    printf "%.23f : %.5f\n", $_, $_ for 0.000005 * 7;; 0.00003500000000000000400 : 0.00004 printf "%.23f : %.5f\n", $_, $_ for 0.000007 * 5;; 0.00003499999999999999700 : 0.00003

    The truncation of a 6 sig.fig constant to a 5 sig.fig. string is the least interesting -- read: pointless -- of them all.

    And using the appropriate rounding mode, not just for the final calculation, but also all intermediate terms becomes very important as the number and complexity of the operations involved increases.

    I stick by my statement that a 64-bit IEEE 754 floating point value is perfectly capable of storing 0.000035 to sufficient accuracy that it can be rounded correctly, for any mathematically significant purpose.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.