in reply to Re^2: Strategy for randomizing large files via sysseek
in thread Strategy for randomizing large files via sysseek

How random do they have to be ?

if you make $linesize * n == $blocksize then you can read a random block of n lines from each of the 50 file handles, randomise these lines in mem, write them out. You only have to mark entire blocks as used. The disk reads will be n times more efficient. Of course lines that were within n of each other will stand a good chance of ending up within n*25 after the randomisation. Perhaps it will be fast enough that you can run two passes and this will be good enough for engineering

Cheers,
R.
  • Comment on Re^3: Strategy for randomizing large files via sysseek

Replies are listed 'Best First'.
Re^4: Strategy for randomizing large files via sysseek
by Anonymous Monk on Sep 09, 2004 at 17:32 UTC
    (OP) They are currently ordered, so they have to be truly random - no chunks allowed, unfortunately.