To get the quickest speed possible, you definitely don't want to load the files into memory. It's probably best to continue reading the file line by line.

If you need to do any operations on the email addresses, such as changing, sorting, or deleteing certain ones.... you should certainly consider a RDBMS.
Depending on your requirements your milage will vary, but in similar situations I've found that using MySQL and DBI proved to be considerably more time efficient than a flat file with tens of thousands (or more) records. I'd be glad to dig up those benchmarks when I get home if anyone is interested, and post them on Monday (right now my only internet connection is at work).