http://qs1969.pair.com?node_id=1036749


in reply to Optimise file line by line parsing, substitute SPLIT

Before we can know whether the extra 8 seconds to use split is of any consequence, we need to know how much other processing you are doing on each line. If the time to process one file is on the order of a minute or so, then extra 8 seconds/file becomes noticeable. If it takes 10 minutes/file, then an extra 8 seconds doesn't make much difference.

If you don't yet know the answer to this question, then it is too soon to make this kind of optimization.

  • Comment on Re: Optimise file line by line parsing, substitute SPLIT