in reply to Re^6: test fails on 64bit uselongdouble Perl
in thread test fails on 64bit uselongdouble Perl

If the desired result is an integer, and the numeric errors are small, there's no need to correctly or robustly round 5.5, so I don't see how your earlier post applies here.
Perl 6 - links to (nearly) everything that is Perl 6.
  • Comment on Re^7: test fails on 64bit uselongdouble Perl

Replies are listed 'Best First'.
Re^8: test fails on 64bit uselongdouble Perl
by ikegami (Patriarch) on Oct 29, 2009 at 22:42 UTC
    Rounding to the nearest integers gives an error of +/- 1.0 even if the actual error is infinitesimally small. The OP said he's not ok with that.
[reply]
      Rounding to the nearest integers gives an error of +/- 1.0 even if the actual error is infinitesimally small.

      I don't believe that. Care to explain?

      You do a calculation, and the result is an integer $n. Due to precision limits you get instead $n+$eps, where abs($eps) is small.

      Why wouldn't proper rounding return $n, but $n+1 or $n-1? At least that's how I understood your statement that it gives an error of +/- 1.0, which doesn't make sense to me at all.

      Perl 6 - links to (nearly) everything that is Perl 6.
[reply]
[d/l]
[select]

        You do a calculation, and the result is an integer $n.

        Number $n. You haven't rounded to an integer yet. 

        $n   = 5.5;
$eps = -0.00000000000001;
printf "%.0f\n", $n;        # 6  expected
printf "%.0f\n", $n+$eps;   # 5  get

        [download]

        If his calculations are suppose to always give integers before rounding, then yes, there's no problem.

[reply]
[d/l]
Re^8: test fails on 64bit uselongdouble Perl
by frankcox (Acolyte) on Oct 29, 2009 at 22:17 UTC

    The result needs to be an integer but the calculation includes floating-point numbers. Again, my problem is that I'm getting different results on 64bit uselongdouble Perl and may best theory currently is differences in floating-point math on different Perls.

[reply]
      Yes, I understood that. And yes, there are differences in floating point handling.

      But I still don't get what's wrong with rounding to an integer - care to explain?

      You currently seem to think that an integral result as an outcome is normal, but as far as I can tell it works only accidentally on some perls, even if that's the majority of perls.

      Perl 6 - links to (nearly) everything that is Perl 6.
[reply]

        Ahhh, I see what you're saying and you make a good point.

        What I'm doing is giving my algorithm a latitude and longitude and it's combining it with some other values it has about a weather data file and it comes back with in integer that's an index into a binary structure of 32bit weather data. My problem comes when I give it a lat/long pair (which may include fractions of a degree), that is exactly one of the pairs that has a data item, I want to get the right index. And I don't always get the right index on all Perls.

        For other lat/long pairs that are between data values rounding is my only option so you're right about that. My algorithm does in fact always round and "accidentally" works on most Perls

        My "problems" started recently when I added support for a new type of file that happened to have several hundred duplicate values. It was a projection to a grid with 180 degree lines on both the east and west side. This let me write a test that asserted that these should be the same value and just one out of 301 failed on uselongdouble. And now here we are ;-)

        (BTW, writing this just gave me an idea which I will post latter... )

[reply]

In Section Seekers of Perl Wisdom