You might find it works better to use BerkeleyDB than a flat file for this. It has a queue mode (basically an on-disk array).