in reply to Re^3: test fails on 64bit uselongdouble Perl
in thread test fails on 64bit uselongdouble Perl

My result needs to be an integer and it needs to be the same integer for all Perls. My problem with uselongdouble Perls is that for some values the result is off by one. Sometimes it's one to big and sometimes one to small.
  • Comment on Re^4: test fails on 64bit uselongdouble Perl

Replies are listed 'Best First'.
Re^5: test fails on 64bit uselongdouble Perl
by ikegami (Patriarch) on Oct 29, 2009 at 20:41 UTC

    Sometimes it's one to big and sometimes one to small.

    Right now, you are truncating the numbers. That means you are introducing an error as big as 1.0

    $ perl -e'printf "%d\n", $_ for 4.9999999999, 5' 4 5

    You need to round instead.

    $ perl -e'printf "%.0f\n", $_ for 4.9999999999, 5' 5 5

    But that's not good enough either.

    $ perl -e'printf "%.0f\n", $_ for 5.4999999999, 5.5' 5 6

    You need to round to your tolerance first.

    $ perl -e'printf "%.0f\n", sprintf "%.8f", $_ for 5.4999999999, 5.5' 6 6
      $ perl -e'printf "%.0f\n", $_ for 5.4999999999, 5.5' 5 6

      I don't get what's the problem with the above. The output is what I would expect.

        The point is that a infinitely small floating point error can change the result by 1.0