Obviously, you need *some* sort of indexing.

I use BerkeleyDB (formerly http://www.sleepycat.com/ - now bought by oracle) with BerkeleyDB to index large flatfiles (on substrings, regexen, whatever).

This will let you keep a setup with the original text files (as you seem to prefer over parsing it to bits into a regular rdbms). Of course, after editing the text file you have to rebuild the index.

Update: BerkelyDB bought by Oracle? Out with it! :) (that data is now in postgresql)