in reply to Re: 15 billion row text file and row deletes - Best Practice?
in thread 15 billion row text file and row deletes - Best Practice?

And bear in mind that sed supports the use of an "edit script" file -- one could take a list of patterns that should be deleted from a file, and turn that into an edit script. Based on the OP's description, the list of serial numbers to kill could be saved as:
/0001234/d /0004567/d /0089123/d ...
If that's in a file called "kill.list", then just run sed like this:
sed -f kill.list big.file > tmp.copy mv tmp.copy big.file
On a file of a few hundred GB, I agree that using sed for this very simple sort of editing would be a significant win (save a lot of run time).