in reply to Re^5: Random data generation.
in thread Random data generation.

Actually, the number of reps allowed is the one parameter that doesn't vary for my scenario. Also, the length of the string will never be less than the size of the set.

The approach you use in salva2 is very interesting, and does make it come out first in every benchmark I've run. One of my own alternatives used a similar approach: Generate a big string completely at random and then pick out a compliant N-char length lump with a regex and return it. Your innovations of a) making the whole string compliant using a regex, b) retaining the buffer across calls; combine to make the approach viable.

The funny thing is that I'm already effectively using the latter part in my code (but not the benchmarks), in as much as I need a bunch of short strings, but I'm calling the generator for one long string and then chopping it into bits using unpack "(A$N)*', gen( $N * 100, @set ). That negates a part of salva2's advantage, and also trades the cost of unpack against the cost of multiple calls to the generator.

Now I have to consider whether your generate a big string and the regex it into compliance approach would still be quicker in-situ. It's not an easy thing to benchmark. Neither is considering the affect it has on the randomness of the result.

It's quite amazing how many variations can result from an ostensibly simple spec. Many thanks for your input.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

Replies are listed 'Best First'.
Re^7: Random data generation.
by salva (Canon) on Jun 28, 2010 at 13:23 UTC
    Neither is considering the affect it has on the randomness of the result

    Well, actually it is!

    Given the $size of the test, the probability of salva2 generating a string starting by two equal characters is 1/($size+1) instead of the 1/$size in the other methods.

    The funny thing is that salva2 is unbiased, they are the other methods that are actually biased!!!

    The following code counts the number of repetitions at any given position in the generated strings: 

    @set = (1..3);
$max_reps = 2;
$len = 10;

my @acu = ([], []);
my @gen = (\&gen_salva, \&gen_salva2);

my ($total, $ds) = (0,0);
my $n = 10000000;
for (1..$n) {
    for (0, 1) {
        my $out = $gen[$_]->();
        while ($out =~ /(.)\1/g) {
            $acu[$_]->[pos($out) - 2]++;
        }
    }
}

use Data::Dumper;
print Dumper \@acu;

    [download]
    And this is the result: 
    salva  => [ 3333044, 2222094, 2591285, 2469706, 2509595, 2496496, 2501
+039, 2501529, 2497584 ]
salva2 => [ 2500639, 2498220, 2499044, 2501878, 2500880, 2500962, 2498
+685, 2499675, 2508077 ]

    [download]

    So, with salva2 the probability of finding a repetition at any position is 1/($size+1) while with the rest of methods the probability varies between the maximum 1/$size (at position 0) and the minimum ($size-1)/$size**2 (at position 1).

[reply]
[d/l]
[select]

      Wow!

      Please excuse my lack of literacy and analysis; but wow! A mathematician that can program. Well, who knew :)

      Sorry to conform to the "Brit stereotype", and complain about the weather, but 10 days of no rain; 4 days of 30C days and 20C+ nights leave me stupefied through lack of sleep. I'll attempt the analysis understanding later when I hopefully will have had more than 60 contiguous minutes of slumber.

[reply]

In Section Seekers of Perl Wisdom