Re: Testing Random Code

I think you're approaching it the wrong way. You need to treat it as a black box and control just the inputs -- in this case, rand.

I wrote Test::MockRandom for exactly this kind of testing. Use it to test your boundary conditions on rand, as that's the only thing that will matter to whether you get all 10 digits. It's much better than playing with srand.

PIN length is almost an invariant because it almost doesn't depend on any input data. You're only at risk of a 5 digit pin (or greater) if the maximum rand value happened to give you "10" -- so I would check length at the same time that I check what happens when the random value (before multiplication) is nearly one.

Update: Here's what it would look like to check the boundary conditions:

use Test::MockRandom 'Some::Other::Package';
use Some::Other::Package 'pin'; # exports pin()

# list of values for rand to use
srand( ( (0) x 4, ( oneish() ) x 4 ); # 0,0,0,0,1,1,1,1

is( pin(), '0000', "rand always zero" );
is( pin(), '9999', "rand always almost one");
[download]

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Comment on Re: Testing Random Code Select or Download Code

Replies are listed 'Best First'.
Re^2: Testing Random Code by BrowserUk (Patriarch) on Sep 19, 2006 at 14:11 UTC
You need to treat it as a black box ... Agreed. But ... PIN length is almost an invariant because it almost doesn't depend on any input data I couldn't disagree more. As `pin()` has no inputs, your assertion is based upon knowledge of what you know is inside the box, which is wrong. For a black box that takes no inputs and produces variable outputs, then only way to test is compare a representative sample of outputs against the specifications for those outputs. Assuming `pin()` is defined to produce a random 4-digit integer; or better `pin()` is defined to produce a random value of type `PIN`, which is (currently) defined as a 4-digit integer; then the tests need to be: Does it always produce a 4-digit integer. Can the full range of integers expected be produced. For large ranges, you would rely upon statistics to sample the distribution, but since this is such a small range, and the generation takes so little time, you might as well saturation test it. If you rely upon knowledge of the implementation to not bother checking for length 4, then a modification by someone not understanding the significance of `"%04d"` that omitted the zero would not be detected. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal? "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l] [select]
Re^3: Testing Random Code by xdg (Monsignor) on Sep 19, 2006 at 15:12 UTC
As pin() has no inputs I'll admit to using "black box" in a poorly defined way -- what I meant was that the function in question isn't really random -- it transforms well defined inputs into outputs. In this sense, pin() takes no arguments, but it does have inputs -- four calls to `rand`. In my view, the right approach is to vary the inputs and examine the outputs -- explicitly test boundary conditions of known inputs rather than trying to evaluate it probabalistically. If you rely upon knowledge of the implementation to not bother checking for length 4, then a modification by someone not understanding the significance of "%04d" that omitted the zero would not be detected. I was referring to the original loop version, actually. It will always concatenate 4 integers. That doesn't vary based on the input. So while it needs to be checked, it doesn't need to be checked everywhere, we only need to check a situation where an input might produce an integer of more than one digit. -xdg Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.	[reply] [d/l]