If possible, make sure both the input and output files are on different physical disks. This alone can sometimes double the throughput. Similarly, avoid using a network filesystem for either file.
bluto
Comment on Re: What's the most efficient way to write out many lines of data?