in reply to Large file processing
Where is your code spending its time? If it's in process_paragraph we need to know the details of process_paragraph. If it's in file I/O there's probably not much you can do. You have already avoided the first trap - slurping a large file into memory. The OS is probably already buffering the file I/O for you so there is unlikely to be much advantage gained by doing your own buffering.
If your paragraphs are fixed length you may gain a little by using read instead of <>, but that gain is likely to be pretty small.
|
|---|