For a single-tasking, (ie. no concurrency issues) application, a well-written, non-DB (RDBMS) solution will always be able to out-perform an RDBMS solution for any given task.

Simple logic. Whatever steps are required to produce a given set of results, all those steps still need to happen regardless of whether you code them yourself, or whether they are performed by the RDBMS on your behalf.

When you code them yourself, you can optimise those steps specifically to the your requirements.

When the equivalent steps are performed by the DB, they will be optimised for the general case. I addition, the RDBMS will also have the overhead of:

  1. Translating SQL (or other data manipulation language) into it's internal form
  2. Optimising the query.
  3. Communications
  4. Locking
  5. Security

However, performance is not the primary reason for storing your data in an RDBMS--all of those overheads are.

Once you store your data in an RDBMS, when a second application comes along that needs to access the same data in a different way, or combine it with other data:

  1. You no longer have to write the code to store, access, search, format or update that data.
  2. You no longer have to optimise that code.
  3. You no longer have to implement, test, debug your own bespoke communications protocols to retrieve your data from remote locations.
  4. You no longer have to implement, test, debug your own locking to achieve concurrency.
  5. You No longer have to implement, test, debug or administer your own security protocols.

These are all very good reasons for using an RDBMS. It's not all sweetness and light. SQL is a pain, normalising data to allow for efficient access for conflicting uses is a (black) art form. Database optimisation is hard, simply because you do not have full control, and there are many conflicting issues that need to be considered.

The very worst reason for considering moving data into an RDBMS is ultimate performance. That's not to say that achieving a reasonable performance isn't possible with an RDBMS. Nor even that, given the amount of effort expended by generally very clever people, over long periods of time, to develop, refine and tune RDBMS performance, that it won't do a better job, more quickly, than most of us will be able to achieve at our first (or even second or third) attempts at a given problem.

It just means that you cannot have all of the additional benefits that you get from using them, without paying some price; and that price is a reduction in ultimate performance.


Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
"Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon

In reply to Re: Basic Perl trumps DBI? Or my poor DB design? by BrowserUk
in thread Basic Perl trumps DBI? Or my poor DB design? by punch_card_don

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.