in reply to Re^4: test fails on 64bit uselongdouble Perl
in thread test fails on 64bit uselongdouble Perl
Sometimes it's one to big and sometimes one to small.
Right now, you are truncating the numbers. That means you are introducing an error as big as 1.0
$ perl -e'printf "%d\n", $_ for 4.9999999999, 5' 4 5
You need to round instead.
$ perl -e'printf "%.0f\n", $_ for 4.9999999999, 5' 5 5
But that's not good enough either.
$ perl -e'printf "%.0f\n", $_ for 5.4999999999, 5.5' 5 6
You need to round to your tolerance first.
$ perl -e'printf "%.0f\n", sprintf "%.8f", $_ for 5.4999999999, 5.5' 6 6
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^6: test fails on 64bit uselongdouble Perl
by kubrat (Scribe) on Oct 30, 2009 at 10:21 UTC | |
by ikegami (Patriarch) on Oct 30, 2009 at 14:48 UTC |