Thanks Ken (and others who responded).
I will try your code and get back with the results.
For others, to address 2 questions asked:
- I am looking at overall process usage as at the end of the day, that is important (OS limits before it kills the process). However, in this simple test, I was not doing any processing.
- I have to slurp the whole file as in the real code the processing time is high and we don't wan to leave the network file handle open for minutes and hours if possible.