in reply to Re: Extract random records from text file
in thread Extract random records from text file

I like this solution best because it does something honestly new and unusual.

However there is one caveat. While it does indeed avoid reading the file into memory, and the actual questions chosen are randomized, the first 10 questions in the list will always appear "in position" if they appear. For perfectly random questions, therefore, it would be best to throw in a shuffle at the end.

  • Comment on Re (tilly) 2 (unrandomness): Extract random records from text file

Replies are listed 'Best First'.
Re: Re (tilly) 2 (unrandomness): Extract random records from text file
by runrig (Abbot) on Oct 02, 2001 at 23:34 UTC
    Or you can just shuffle them from the start:
    my $num = shift;
my @init = (0..$num-1);
my @questions;
while (<>) {
  $questions[splice @init, rand(@init), 1] = $_, next if $. <= $num;
  my $i = rand($.);
  $questions[$i] = $_ if $i < $num;
}

    [download]
[reply]
[d/l]
      I prefer shuffling at the end using Fischer-Yates. Shuffling at the start with splice is algorithmically very bad. OK, with only 10 elements, it works just fine, but I tend avoid splice on general principle if there is a more efficient alternative. (Which there is in this case.)
[reply]

In Section Seekers of Perl Wisdom