tritan has asked for the wisdom of the Perl Monks concerning the following question:

Hey all,

I've recently been trying to update my understanding of perl's rand() function and the improvements / better implementations available. I believe I understand why using Math::Random::MT is a better idea overall. But, given my own minimal mathematical background, I cannot figure out when the MT implementation is 'not enough'. I'm also not sure if this is something I should ever worry about (I'm not really doing complex statistical modeling -- I'm just curious).

So, my questions are 1) is there an upper limit on the usefulness to Math::Random::MT, where another algorithm is better? 2) Are there even any better approaches, currently, than Math::Random::MT in perl?

Edit: By 'better', I mean when the algorithm begins to break down in its ability to be random. Or when the values begin to repeat one after another after enough iterations. I'm not really interested in relative speeds or security atm for this question.

Edit2: Thanks everyone for replying. I think I've got a lot more information now, and some of the answers I was looking for.

Thanks
  • Comment on Upper limit to Mersenne Twister Algorithm?

Replies are listed 'Best First'.
Re: Upper limit to Mersenne Twister Algorithm?
by moritz (Cardinal) on Aug 30, 2010 at 16:37 UTC
    As with nearly everything, it depends on your use case.

    If you want to generate transaction numbers for online bank transfers that you give your customers, using any kind of pseudo random number generator is probably a bad idea (there are hardware devices that generate truely random numbers based on optical quantum effects).

    If you want to write some simulations for which the correlation length of the build-in PRNG in Perl isn't sufficient, the Mersenne Twister is a probably a very good alternative.

    Generation of pseudo random numbers is a big area of research, and I'm not very familiar with it; but my impression is that if you are willing to trade speed and/or memory, you can get even better PRNGs. But you should only bother if your use case requires it.

    Perl 6 - links to (nearly) everything that is Perl 6.
Re: Upper limit to Mersenne Twister Algorithm?
by Your Mother (Archbishop) on Aug 30, 2010 at 16:33 UTC
      Wow, interesting. Thanks for the link : )
Re: Upper limit to Mersenne Twister Algorithm?
by BrowserUk (Patriarch) on Aug 30, 2010 at 16:37 UTC

    It very much depends upon your use.

    Mersenne Twister provides a very good PRNG, but it is, with sufficient samples, predictable. So, they say, this is no good for cryptographic purposes.

    For other purposes, the CPAN implementation is excellent, but slow. It is also limited to 32-bits.

    There is a 64-bit C version.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Upper limit to Mersenne Twister Algorithm?
by JavaFan (Canon) on Aug 30, 2010 at 16:21 UTC
    You talk about "better". Better in which sense? Without stating what you want of a random number generator, it's impossible to order algorithms on a scale of "good, better, best".

    Things you may want: speed, repeatability, hard to predict, etc.

      Updated! Does my question make more sense now?

        Not at all. In fact, it even makes less sense. What do you mean by "breaks down in its ability to be random"? Note that for every PRNG, given the algorithm, seed and sequence number, the rest of the sequence is 100% predictable.
Re: Upper limit to Mersenne Twister Algorithm?
by baxy77bax (Deacon) on Aug 30, 2010 at 17:16 UTC
    hi,

    as people above mentioned you have to know for what you need the generator for. my personal opinion is as far as the software type random number generator goes, all of them are a joke if meant to be used for some real cryptography. rand() function in perl is inhereted from libc that has a RAND_MAX from 2**15 - 2**31 which is also interesting because since perl doesn't handle big integers very well by default, you need to use bigint.

    also i read somewhere that the authors of rand() function guarantee that the number you get is a random but they don't guarantee that more of them are random - which implies that there is a pattern within the whole system.

    to me regular MT with a period of 2**k-1 , k = 19937, is more than enough for my simulations. but i can't say if it will satisfy your requests. my former coworker had this problem with MT and some other algorithms (the period wasn't enough for him), so he turned to QRBG. now, this is a machine based generator that is nothing like software generators and it guaranties the randomness of one number or more of them. so my advice is : start worrying when the problem emerges, until then use rand() build-in function.

    cheers