I wish I could post those files, but they are too big (about 20GB)
The files wouldn't help us to help you, but the code you are using to read them would.
Some possibilities are:
- You are running out of file handles;
- you are running out of memory;
- your OS/user account is set up to kill your processes if the exceed certain limits of cpu/IO.memory usage;
- many others.
If you posted the code, most of those could be eliminated very easily and the real reason might well be obvious to someone here.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".