Re: Which is quicker - writing to a file or inserting into a db?

Writing to a plain file is probably faster. Talking to a database has some overhead like passing data through a network/socket connection, parsing and interpreting SQL statements (although excessive parsing can be avoided by using a prepared statement). Also there's additional overhead in the underlying file structure of databases, and possible the index overhead.

Comment on Re: Which is quicker - writing to a file or inserting into a db?

Replies are listed 'Best First'.
Re^2: Which is quicker - writing to a file or inserting into a db? by Anonymous Monk on Dec 13, 2007 at 23:39 UTC
If I'm not checking whether the emails are duplicates or not (meaning duplicate emails are getting appeneded to the file)...does it matter that the files keeps getting bigger and bigger? Will that affect performance at all? I was thinking of appending to the file and then when the time comes just hashing everything to find the unique emails.	[reply]
Re^3: Which is quicker - writing to a file or inserting into a db? by eserte (Deacon) on Dec 14, 2007 at 00:12 UTC
There's no performance penalty in appending to a large file. But you should probably think about locking, as multiple processes simultaneously writing to the same file may cause problems. Or just make sure that each process writes to its own file, e.g. by appending $$ to the file name.	[reply]
Re^2: Which is quicker - writing to a file or inserting into a db? by spatterson (Pilgrim) on Dec 14, 2007 at 16:37 UTC
Don't forget about file locking, a datbase can typically handle (with some caching) simultaneous writes to the same table, but simultaneous writes to a single text file from different processes is likely to fail. just another cpan module author	[reply]