Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Very small numbers

by Win (Novice)
on Feb 09, 2004 at 13:31 UTC ( [id://327621]=perlquestion: print w/replies, xml ) Need Help??

Win has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

Can anyone point me towards overcoming this problem. When using the fdist function of the Statistics::Distribution module I get the error message:

Can't take log of 0 at ...........
I guess that I may need to fiddle with the module's code to overcome this. For smaller numerator figures I get the result I need, in line with Excel.

Can anyone give me any pointers?

Replies are listed 'Best First'.
Re: Very small numbers
by kvale (Monsignor) on Feb 09, 2004 at 17:49 UTC
    I think we need nore detail on your problem to give a useful answer. What numbers are causing this problem? What distribution function are you trying to evaluate? Do you mean fdistr? Show us your code.

    If your numbers are so small that they are causing arithmetic underflow of your floating point represenentation, then you need to be more clever in how you deal with them. A general technique is to work with the log of a very small number, rather than the number itself. Then multiplications become additions, etc.

    Another approach is derive a perturbative expansion of your probabilitiy distribution for very small numbers and use that directly.

    -Mark

      I think that I will have to do something along the lines of:

      if ($numerator > 7169 || $denominator > 1650){ print "Do nothing"; # Place another method in here } else { my $fprob = Statistics::Distributions::fprob($numerator,$den +ominator,0.025); }

      The FINV function in Excel (equivalent to the fprob function in Statistics::Distributions) is able to go above the denominator and numerator values specified as limits here.

        Sorry, please replace fprob with fdistr in my previous post.
Re: Very small numbers
by belg4mit (Prior) on Feb 09, 2004 at 16:13 UTC
    One thing you might try (though this may also fail due to floating point errors) is multiplying your number by a few hundred before taking the log. Then subtracting from the result to account for the initial magnification.
    log(a*b) = log a + log b log10 1000 = 3 log10 6 = 0.77815125 log10 6000 = 3.77815125 ...
    UPDATE: Although this makes me think we ought to have a Math::SmallRat and SmallFloat. PDL or Math::FixedPrecision might also work.

    UPDATE: Apparently Math::BigFloat is just a bit of a slight misnomer. It's not just for BIG numbers. The Big imples a large number of bits assigned to store the number.

    --
    I'm not belgian but I play one on TV.

      Well, if your number is small enough that Perl thinks the number is 0, multiplying it with 100, or whatever number you choose, will not do you any good:
      $ perl -wle 'print log (10**-4950)' -11397.7067720196325 $ perl -wle 'print log (10**-4951)' Can't take log of 0 at -e line 1. $ perl -wle 'print log (10**-4951 * 100)' Can't take log of 0 at -e line 1.

      Abigail

      PDL unfortunately cannot help with this, since it (currently) only works with C types, in this case probably a double-precision floating point number. However, the techniques discussed elsewhere in this thread to avoid underflow (especially scaling) will work great to overcome that.
      Maths::BigFloat is not available via ppm.
        The name is Math::BigFloat, and it is included with perl (though if you have an old version of perl, you may want to update it since there have been bugfixes).
Re: Very small numbers
by Abigail-II (Bishop) on Feb 09, 2004 at 13:48 UTC
    Well, you can't take the log of 0, as that's undefined. I'm not sure what you real question is. Do you want to take the log of 0 anyhow? Does it surprise you the package is taking the log of 0? Do you want the package to determine it's about to take the log 0, and use some other calculation?

    Abigail

      It surprises me that it is trying to take a log of zero. I guess that the module is generating a very small float and is calling it zero.
Re: Very small numbers
by RandomWalk (Beadle) on Feb 11, 2004 at 01:19 UTC
    Dear Win,

    I you look at the definition of the F-distribution you'll see a ratio of gamma functions. Computing gamma(200) will overflow most engines, so what you need is some indirect method for computing the ratio.

    Now if you read the docs you'll see that Statistics::Distributions is not intended for high precision work (though your issue isn't exactly precision) and that the authors direct you to a website for more sophisticated algorithms.

    You might also reconsider why you need to be evaluating the F-distribution at such extreme parameters. Maybe you can save yourself the trouble of dealing with asymptotics.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://327621]
Approved by DaWolf
Front-paged by broquaint
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (7)
As of 2024-03-28 18:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found