in reply to Re: RFC: OtoDB and rolling your own scalable datastore
in thread RFC: OtoDB and rolling your own scalable datastore

I think the argument for denormalisation is something like this: If you have a DB that is mostly read from (and more importantly rows are rarely updated), with lots of data and many concurrent requests, denormalisation is a way to improve performance by avoiding the overhead of a join. But as you point out, the price is added complexity when updating.

It's a useful trick, but maybe not as useful as some think. It is, after all, just another way to optimise performance.

  • Comment on Re^2: RFC: OtoDB and rolling your own scalable datastore

Replies are listed 'Best First'.
Re^3: RFC: OtoDB and rolling your own scalable datastore
by CountZero (Bishop) on Jul 15, 2008 at 19:23 UTC
    It is a trick used in certain data-mining applications. You denormalize your "live" data into a number of additional tables and you do your data-mining queries on these denormalized tables. It is very fast as there are much less JOINs to be done when querying the data, but you are limited in what you can query. If you do this denormalization as a batch job when the database server is not very busy (say every night or week-end) and put the additional tables on another server optimized for read-access, I can see the benefit.

    The drawback is that the data becomes stale as soon as the live server updates and that not all queries are possible because you see the data through the mirror of the denormalization process. Sometimes this is even considered a benefit: "Now you can get at your data without having to write SQL." Yeah sure what a benefit ...

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James