Re^2: Testing Random Code

You need to treat it as a black box ...

Agreed. But ...

PIN length is almost an invariant because it almost doesn't depend on any input data

I couldn't disagree more.

As pin() has no inputs, your assertion is based upon knowledge of what you know is inside the box, which is wrong. For a black box that takes no inputs and produces variable outputs, then only way to test is compare a representative sample of outputs against the specifications for those outputs.

Assuming pin() is defined to produce a random 4-digit integer; or better pin() is defined to produce a random value of type PIN, which is (currently) defined as a 4-digit integer; then the tests need to be:

Does it always produce a 4-digit integer.
Can the full range of integers expected be produced.
For large ranges, you would rely upon statistics to sample the distribution, but since this is such a small range, and the generation takes so little time, you might as well saturation test it.

If you rely upon knowledge of the implementation to not bother checking for length 4, then a modification by someone not understanding the significance of "%04d" that omitted the zero would not be detected.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re^2: Testing Random Code Select or Download Code

Replies are listed 'Best First'.
Re^3: Testing Random Code by xdg (Monsignor) on Sep 19, 2006 at 15:12 UTC
As pin() has no inputs I'll admit to using "black box" in a poorly defined way -- what I meant was that the function in question isn't really random -- it transforms well defined inputs into outputs. In this sense, pin() takes no arguments, but it does have inputs -- four calls to `rand`. In my view, the right approach is to vary the inputs and examine the outputs -- explicitly test boundary conditions of known inputs rather than trying to evaluate it probabalistically. If you rely upon knowledge of the implementation to not bother checking for length 4, then a modification by someone not understanding the significance of "%04d" that omitted the zero would not be detected. I was referring to the original loop version, actually. It will always concatenate 4 integers. That doesn't vary based on the input. So while it needs to be checked, it doesn't need to be checked everywhere, we only need to check a situation where an input might produce an integer of more than one digit. -xdg Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.	[reply] [d/l]

Replies are listed 'Best First'.

Re^3: Testing Random Code
by xdg (Monsignor) on Sep 19, 2006 at 15:12 UTC

As pin() has no inputs

I'll admit to using "black box" in a poorly defined way -- what I meant was that the function in question isn't really random -- it transforms well defined inputs into outputs. In this sense, pin() takes no arguments, but it does have inputs -- four calls to rand. In my view, the right approach is to vary the inputs and examine the outputs -- explicitly test boundary conditions of known inputs rather than trying to evaluate it probabalistically.

If you rely upon knowledge of the implementation to not bother checking for length 4, then a modification by someone not understanding the significance of "%04d" that omitted the zero would not be detected.

I was referring to the original loop version, actually. It will always concatenate 4 integers. That doesn't vary based on the input. So while it needs to be checked, it doesn't need to be checked everywhere, we only need to check a situation where an input might produce an integer of more than one digit.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

[reply]
[d/l]