Boldra has asked for the wisdom of the Perl Monks concerning the following question:

I was reading madermans post about Composing a SQL statement and I was reminded of some very peculiar software architecture I saw last year. As I started writing this I realised I had gone way offtopic, but I still wanted the monastary's opinion.

This system I was working on had introduced 'SQL abstraction layer' between the database functions layer and the database itself:

webserver
CGI layer
processing
DB functions layer
SQL abstraction layer
DBI/DBD
database

The DB functions layer defined commonly used functions like 'add_user' 'add_transaction'. Such layers are common, and they allow you to cleanly change the database backend without worrying about the front. But in this case the DB functions layer didn't directly use SQL... Instead it called from an array of new functions derived from the level below, eg:

&my_db_connection->update( table_name => $table_name, keys => &my_db_connection->select( table_name => $table_name, return_field => 'userID', select_field => 'userName', value => $username), update_field => 'userAddress', update_value => $new_address);
The select and update methods would generate and execute the SQL. I was expecting to find a more direct generation of SQL, eg:
$sql = "UPDATE users SET userAddress='$new_address' WHERE userName='$ +username'"
The only advantages of this approach (IMHO) are: Whereas the disadvantages are: And of course all of the other issues that arise simply by virtue of writing an extra 2000 lines of code

Has anyone got ANY ideas why the original developers might have done this ? ? ?

Replies are listed 'Best First'.
Re: Architecture of a Database system
by arhuman (Vicar) on Apr 04, 2001 at 17:25 UTC
    First about the advantages :

    The only advantages of this approach (IMHO) are:
    Security - malicious code has to navigate an extra layer of obscurity


    Ahem... Security through obscurity isn't security...


    Safe development - Once the abstraction layer is written,
    developers working on higher levels are limited in the damage they can do
    (can't "DROP TABLE")


    You can achieve the same result without the SQL abstraction layer...
    All you need is to provide a restricted API ( in your case the DB function layer)


    Now about the disadvantages :

    I'd just like to point out that the the Database locked to code problem
    isn't bound to the architecture IMHO.
    I suppose that even with this architecture
    a clean coding could produce a relative database independance
    (using list/hash to manage columns...)

    Now to answer why, I have several guesses :
    • Lack of impatience (too much abstraction here, to my mind)
    • Some people think that complex means smart...
    • The bad implementation syndrom :
      Or "how a good idea (database independance) could lead to an absurd implementation"
      By duplicating existing mechanisms(database independant API), and allowing some limitations that ruin the whole benefit(the column dependance problem))
    • Megalomaniac Lead Programmer:
      You know the one who uses a lot of buzz words (and a lot of accronyms : DFL/SAL for DB functions layer and SQL abstraction layer ;-)


    "Only Bad Coders Badly Code In Perl" (OBC2IP)
Re: Architecture of a Database system
by Masem (Monsignor) on Apr 04, 2001 at 17:16 UTC
    I'd suspect that while you list it as a misfeature, the abstractioning away of SQL might have been considered a feature for this developers. Despite SQL's simplicity, someone up the management branch might have decided to avoid introducing a new 'language' to pure perl programmers.

    It could also easily be a result of Object Orientied-itis. Such a scheme as you describe EASILY fits itself into someone making a class heirarchy of web sites.

    In either case, inexperienced perl programmers or higher-ups non-programmers sound like the root cause.


    Dr. Michael K. Neylon - mneylon-pm@masemware.com || "You've left the lens cap of your mind on again, Pinky" - The Brain
      It could also easily be a result of Object Orientied-itis.

      It's been my experience that most attempts to build abstraction layers on top of SQL are initiated by the programmers themselves, often for just this reason. They (uh, "we", since I've done this too :-) see SQL as lacking in OO goodness, and want to impose our own abstraction atop it.

      Perhaps this one of of those cases where there's a "I've implemented my own _____ abstraction" club. The existence of such a club would explain a lot of strange code.

Re: Architecture of a Database system
by voyager (Friar) on Apr 04, 2001 at 17:33 UTC
    Dittos to what Masem says. This actually looks like the first system I designed immediately after reading some OO books. (In this case that is not a complement.)

    This is a bit of an over simplification, but the design should be for change. Installed applications change, and requirements change over the development cycle. Change should be easy. So if a SQL table changes (column add/change/remove), or the tables are completely redesigned, how easy is it to change the application?

    If a simple table change causes changes in all most of your layers that might indicate a design problem. I like the idea of generating columns lists from the db, but is it done at all levels?

    Having a new column automatically added to an INSERT statement is great, but is it automatically added to the web form? If so, is the layout what you want? If not, how does the database ever get a value? And do you want to be automatically generating column lists in one part of your application, but not others?

    To make a long story longer, given that it is in place already, the added complexity probably won't hurt you as long as you have the ability to hand-code the SQL statements that don't fit neatly into the schema. I actually do something similar for the maintenance part of my apps that only admin users see. But for end-users, presentation and details matter so much that hand-coding is more normal.

Re: Architecture of a Database system
by grinder (Bishop) on Apr 04, 2001 at 17:54 UTC

    I posted a snippet a while ago that shows one method of generating an SQL INSERT statement from an anonymous hash and a table name, which is about as simple as you can get. Plus it does clever things to produce placeholdered statements and batched commits.

    It really is a funky piece of code that serves me well. OTOH it only works well in a limited domain, that is, it is not the ideal solution when you are inserting different columns sets at different points in the program (as it must generate a fresh prepared statement for each call and it's up to you to remember to tell it to do so).

    So I think it would be hard to come up with something that satisfies the general case.

    Plus, my main aim was to minimise characters. Not in a golf TM sense, but I didn't want to have to dick around with too many braces, parens or other syntactic fluff.

    An example call looks like this:

    insert 'aglrelvalue', { att_value => $id_mission, attribute_id => B0, client => DEFCLIENT, last_update => $mis_update, percentage => 100.00, rel_attr_id => YC, rel_value => $regvp, user_id => USER, };

    --
    g r i n d e r
      what you have done is re-implement the DBIx::Recordset Insert() API function.
        Man I hate when that happens.

        --
        g r i n d e r
Re: Architecture of a Database system
by suaveant (Parson) on Apr 04, 2001 at 17:27 UTC
    It seems like a good way to make it as simple as possible to change databases in the future... theoretically you could just change the abstraction layer. Granted in most cases you do that in the DB functions layer... but I suppose the SQL abstraction layer encapsulates the SQL specific stuff even more... theretically making it easier to change... since all SQLs are certainly NOT created equal :)
                    - Ant