Re: Where is the bottleneck in flat file databases?

Scalability, maintaince and integrity.

Scalability Unless you've got a very tightly scoped project, the inherent creep in data model will invariably lead to your data set growing. If you want to follow any type of normalisation of your data it means having multiple text files. This type of design is not going to scale very well.

Maintaince Adding records, removing records and maintaining the integrity of your record structure can be difficult. You've got two clients requesting access to the same record, both writing, which should go 1st? we're talking about locking. How do you manage backups? restore a complete file? what if its been updated? we're talking transaction logs..

Integrity What character do you use as a delimiter? and what if a user wishes to use it for a legit purpose? what about a data model which says only values 'a, b and c' are valid, and a user enters d? This kind of things can be handled very easily using foreign key constraints in an RDBMS.

Any serious designer will tell you an RDBMS is the best way to go in terms of building data models (assuming you're data model is relational). You've got all the access control, the transactions, the ability to fundamentally normalise your data, its all there, and, its all available freely.

Now all the negative stuff is out of the way, there are instances where a FF may be handy. Read only as MarkM pointed out is probably the best use of flatfiles.

Comment on Re: Where is the bottleneck in flat file databases?

Replies are listed 'Best First'.

Re: Answer: Where is the bottleneck in flat file databases?
by Wysardry (Pilgrim) on Jan 24, 2003 at 00:20 UTC

I agree with your points against using flat file databases, but in this case those drawbacks will have little effect.

In this case I'm hoping to improve on an existing program that is widely used to create web sites. As such, the schema is unlikely to change radically once defined.

The database will only ever be modified by one person, so there shouldn't be any problems with multiple users writing to the same file. Backups will be made each time a record is modified as described in my reply to MarkM.

Which character to use as a delimiter can be define in a separate configuration file. As the data will be use to create web pages, if the user legitimately needs to use the same character in the text of their page they can use an HTML entity instead.

I agree that a dedicated RDBMS is usually the way to go, but not everyone has the luxury of having access to one (many free hosts still don't offer MySQL).

Text files can be an advantage to this system if one or more of the following applies:-

The user is on a budget
A suitable RDBMS is not available
The user has little experience with using databases
The data is only ever changed by a single person
The user can only execute the script via a hosting service

Storing the data in plain text files is a big advantage in the latter case, as the user has the option to modify the database using a standard text editor.

__________
"Every program has at least one bug and can be shortened by at least one instruction -- from which, by induction, one can deduce that every program can be reduced to one instruction which doesn't work." -- (Author Unknown)

[reply]


Come for the quick hacks, stay for the epiphanies.
	PerlMonks