Re: test fails on 64bit uselongdouble Perl

This discussion has helped me work out this issue in my mind. I'll be reworking my algorithm and I have a better idea now about how.

What I need to do is take a latitude/longitude pair, which may include fractions of a degree, and return an integer index into a binary data structure in a GRIB weather data file. I've been worrying about rounding to the nearest int but really I need to round to the nearest data point. The data points, projected onto a lat/long grid, are centered in a rectangle LaInc tall and LoInc wide (from my code above). These increments can also include fractions. All, I have to do is figure out which rectangle a given lat/long pair is in and return a number based on the scanning order for the type of file it is. (The above code works for west-east and south-north scanning.)

uselongdouble really doesn't have much to do with this except that that's what whacked me up side a' the head. So, thanks for that, and thanks CPAN Testers!

Comment on Re: test fails on 64bit uselongdouble Perl

Replies are listed 'Best First'.
Re^2: test fails on 64bit uselongdouble Perl by frankcox (Acolyte) on Oct 30, 2009 at 23:49 UTC
I tested the advice of ikegami here and changed my code to round floating-point numbers rather than truncate as I was inadvertently doing. That is, I changed: `return sprintf "%d", $out; # Wrong, truncates.` [download] to: `return sprintf "%.0f", sprintf "%.6f", $out;` [download] With this change I now see consistent results on all Perl platforms including 64bit uselongdouble ones. Thanks to everyone who helped out here! I apologize for being a little slow to see the light but the end result is good. I still plan to rework my algorithm but now I can do that as part of a regular release rather than as part of a bug fix.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^2: test fails on 64bit uselongdouble Perl
by frankcox (Acolyte) on Oct 30, 2009 at 23:49 UTC

I tested the advice of ikegami here and changed my code to round floating-point numbers rather than truncate as I was inadvertently doing. That is, I changed:

return sprintf "%d", $out; # Wrong, truncates.
[download]

to:

return sprintf "%.0f", sprintf "%.6f", $out;
[download]

With this change I now see consistent results on all Perl platforms including 64bit uselongdouble ones. Thanks to everyone who helped out here! I apologize for being a little slow to see the light but the end result is good.

I still plan to rework my algorithm but now I can do that as part of a regular release rather than as part of a bug fix.

[reply]
[d/l]
[select]