in reply to Re^9: test fails on 64bit uselongdouble Perl
in thread test fails on 64bit uselongdouble Perl

Ahhh, I see what you're saying and you make a good point.

What I'm doing is giving my algorithm a latitude and longitude and it's combining it with some other values it has about a weather data file and it comes back with in integer that's an index into a binary structure of 32bit weather data. My problem comes when I give it a lat/long pair (which may include fractions of a degree), that is exactly one of the pairs that has a data item, I want to get the right index. And I don't always get the right index on all Perls.

For other lat/long pairs that are between data values rounding is my only option so you're right about that. My algorithm does in fact always round and "accidentally" works on most Perls

My "problems" started recently when I added support for a new type of file that happened to have several hundred duplicate values. It was a projection to a grid with 180 degree lines on both the east and west side. This let me write a test that asserted that these should be the same value and just one out of 301 failed on uselongdouble. And now here we are ;-)

(BTW, writing this just gave me an idea which I will post latter... )

  • Comment on Re^10: test fails on 64bit uselongdouble Perl

Replies are listed 'Best First'.
Re^11: test fails on 64bit uselongdouble Perl
by ikegami (Patriarch) on Oct 29, 2009 at 23:38 UTC

    My algorithm does in fact always round

    As I mentioned a couple of times already (1, 2), you showed it to be truncating (not rounding).