nickschurch has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to read in a sequence of numbers like 50137.685706 from a file into a piddle. When I do this the number, despite being read in as a 'double' comes out truncated to 50137.686. How do I read in this number without any truncation or rounding? Is there a 'bigger' datatype than double?

Replies are listed 'Best First'.
Re: PDL datatype
by tachyon-II (Chaplain) on Nov 28, 2007 at 03:39 UTC
[reply]
Re: PDL datatype
by locked_user sundialsvc4 (Abbot) on Nov 28, 2007 at 04:19 UTC

    If you ask the computer to convert a sequence of bytes such as "50137.685706" (a total of 12 characters consisting of the digits 0-9 plus '.') into any sort of binary representation ... and back again, into yet-another series of decimal characters ... then by-definition there must be two consecutive character-to-binary and binary-to-character conversions.

    Therefore, I ask you:   what do you have, byte-by-byte, and what do you ultimately want to do with it?

Re: PDL datatype
by moritz (Cardinal) on Nov 28, 2007 at 06:40 UTC
    Chances are that the input is read as it should, but it's not displayed with sufficient accuracy.

    See printf and sprintf for more details on how to print numbers.

[reply]
      Its definately not just s 'print to screen' memory problem. They are actually being truncated when they are read in.
      What I want to do, as part of a large piece of scientific analysis, is search through is search through a table picking out any rows in that table that have a timestamp matching the timestamps in a different table. The timestamps are, by necessity, very precise.
      I've got round it by reading the numbers before the decimal and after the decimal as different columns and then pattern matching - but its not very elegant...
[reply]
Re: PDL datatype
by dwm042 (Priest) on Nov 28, 2007 at 12:55 UTC
    Hate to ask, almost, but I'd love to see some example code. Then we'd have something to test with.
[reply]
Re: PDL datatype
by moklevat (Priest) on Nov 28, 2007 at 16:27 UTC
    Do you get the same truncation when creating piddles in the perldl shell? On two separate systems (Windows and Linux) I get: 
    perldl> $x = 50137.685706

perldl> print $x
50137.685706

    [download]
    I guess I second (or third) the request to see some code. Is it possible you are truncating somewhere upstream of PDL?
[reply]
[d/l]
      Hmmmm... thats interesting moklevat.... When I do what you did I get a non truncated numbers! I did a few tests and take a look at this:
      [njs@localhost Shu_Hardness_Ratios]$ perldl
perlDL shell v1.33
 PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
 'COPYING' in the PDL distribution. This is free software and you
 are welcome to redistribute it under certain conditions, see
 the same file for details.
ReadLines, NiceSlice, MultiLines  enabled
Reading PDL/default.perldlrc...
Found docs database /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thre
+ad-multi/PDL/pdldoc.db
Type 'help' for online help
Type 'demo' for online demos
Loaded PDL v2.4.3

Note: AutoLoader not enabled ('use PDL::AutoLoader' recommended)

perldl> $x=50137.685706
perldl> print $x
50137.685706

perldl> $x=pdl(50137.685706)
perldl> print $x
50137.685706

perldl> $x=zeroes(1)
perldl> use PDL::NiceSlice
perldl> $x(0) .= 50137.685706
perldl> print $x
[ 50137.686]

perldl> $x = rcols "bursts.txt"
Reading data into piddles of type: [ Double ]
Read in  33  elements.
perldl> print $x
[ 50093.199  50094.308  50954.953  51767.383  51958.219  52005.696  52
+005.763  52142.188  52422.043  52459.414  52482.509  52693.953  52752
+.918  52836.803  52837.198  53107.974  53125.429   53141.53  53198.49
+5  53243.578  53489.381  53603.709   53661.37  53724.982  53724.983  
+53864.724  53882.002  53969.248  54224.674  54225.636  54304.474  543
+04.539  54366.513]

      [download]
      So if you type the number in straight its no problem, but if you read it from a file using rcols, or if you define a piddle and then put the number into the pre-existing piddle, then it truncates the number! I wonder why it does this?
      In the PDL manual 'double' is the biggest data type listed so I on;y really understand why creating a piddle with this number in works one way and not another - or how to fix it.
      I'd love to give you an example of the previous code it was failing in but I've re-written that with my aforementioned inelegant work around.
[reply]
[d/l]
        PDL::Core (which stringifies ndarrays) uses these variables: 
        $PDL::floatformat  = "%7g";    # Default print format for long numbers
$PDL::doubleformat = "%10.8g";

        [download]
        And when it is stringifying, as you asked it to do, it uses the values of those. The above is the current (2.080) values, but those have been the same since 2.002 back in 1999. However, printf's g format treats the precision (the number after the ".", here 8) like so:
        The precision specifies the number of significant digits.
        50137.686 has 8 significant digits. To change this behaviour, just assign a different value to e.g. $PDL::doubleformat.

        The way I checked the above was my usual technique: use git log -p -w and text-search for a relevant string, here "floatformat".

[reply]
[d/l]
[select]
        It turns out it may not be so interesting now that my head is on a bit straighter. Assigning $x=50137.585706 does not create a piddle, as I would have known if I had bothered to print info($x).

        I previously looked into adding a new types when the topic of using BigInts in piddles came up a while back, but I had to get back to my actual work. It may be that creating a new type would get you where you want to go.

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom