in reply to Merging Many Files Side by Side
You could save some memory consumption (and perhaps reduce slowdown due to memory paging) if you opened the files one by one. By opening all the files at once, you are also opening 200 buffers for each of the files all at once.
With files that large, you might also want to consider tying the arrays holding the columns to random access files. I once had a ten-fold speed up (in a C++ program) just by using temp files instead of RAM to store data while I was processing. See Tie::File
If your columns are fixed width, you might be able to to avoid two cycles, one to read the array and one to print the array, by keeping a variable that stores the current line length. Then instead of an array of arrays, you could just:
Best, beth
|
|---|