If the number-of-lines in each file is ... say ... “less than a few tens of millions,” then a reasonable first step might be to shove both files into memory...
My understanding from the original post is that there is only one file, not two.
This makes the algorithm quite different.
And if the lines are sorted by the unique identifier as they appear to be, there is not need to load any file into memory. You only need to maintain a min and a max for each identifier.
| [reply] |
I am trying to read through this file line by line and then for the first line that has IMB and 060410. I want to find the min number and the max number and output it in one line like this
IMB,060410,V1 ,371094378,371096486
Then I have another unique number which is IMB,107951
I need to read through the lines and have one line output
IMB,107951,V1 ,981157588,981176199,1098
| [reply] |