Re^2: Random Math Question

Dominus,
So the question I think you need to ask here is whether a sequence of arrays shuffled by your method would be distinguishable from a sequence of arrays shuffled into truly random orders.

Oh, I totally agree now that I have had a night's rest and a few good replies. It was one of those things where, in trying to make a convincing argument, I lost sight of the forest through the trees. As I indicated in another reply, I considered changing the method to change what elements appeared in each group for every iteration. Without that change, it should be easy to see that the re-ordering isn't random.

I don't mind being wrong for the sake of a good discussion.

Cheers - L~R

Comment on Re^2: Random Math Question

Replies are listed 'Best First'.
Re: Random Math Question by Dominus (Parson) on Oct 11, 2005 at 13:48 UTC
Said Limbic~Region: As I indicated in another reply, I considered changing the method to change what elements appeared in each group for every iteration. Without that change, it should be easy to see that the re-ordering isn't random. As I kept saying on IRC, it doesn't matter how you change the method, because the results won't be random regardless of what method you choose. If you only have enough entropy to randomize 8 things, and you try to generate random permutations of 64 things, you are not going to be able to generate all possible permutations. You asked for a method that I could use to tell that your method wasn't working. Here's one: give me a sample of 200,000 of the "randomized" 64-element lists. You said "(Assume we have 64 elements in our list but only enough entropy to truly randomize 8)." Then I can tell there's something fishy because the sample you gave me will have lots of duplicate lists in it. You have only about log(8!) = 15.3 bits of entropy, so in the 200,000 items, there will be several that appear five times each. But the probability of such an occurrence in a truly random sample of 200,000 shuffled lists is about 10^-356. So it'll be totally obvious that you're cheating. This will be true regardless of what clever method you are using to mix up the items. You can't get blood from a stone. As I said last night, what you're doing is like taking a 16-bit PRNG and then using it to generate 32-bit numbers by concatenating two 16-bit numbers together. If you hand someone a list of your "random" 32-bit numbers, it will be clear that they aren't really random. -- Mark Dominus Perl Paraphernalia	[reply]
Re^2: Random Math Question by Limbic~Region (Chancellor) on Oct 12, 2005 at 13:49 UTC
Dominus, I think you read more into what I was saying then I intended. Emphasis should have been on the fact that the process I outlined would be trivial to recognize as not being randomly distributed. The point of my reply was to say I recognize that not only was the process I outlined flawed but it wasn't even a good fake. Changing the process could make it harder to detect but that wasn't the point of my reply. Your proposal for determining the fake is in alignment with other responses. Unfortunately, it requires multiple lists to be generated where I was asking for only 1. 1 list is obviously not statistically valid so my question was flawed. For the benefit of others I am going to characterize the problem one more time but rest assured I have no argument with what you are saying. Read more... (1249 Bytes) Cheers - L~R	[reply]