in reply to Randomize lines with limited memory

If you don't really need a fair shuffle, you could get around the memory limitations by simply process the file by parts. Do a fair shuffle on each of the parts, and cat them together in random order.
For better results, do this a number of times using a non-zero offset.

Of course, you won't have a real fair shuffle, but maybe this is good enough.

Hope this helps, -gjb-

  • Comment on Re: Randomize lines with limited memory