in reply to What do people want from DB abstraction modules?

  • Comment on Re: What do people want from DB abstraction modules?

Replies are listed 'Best First'.
Re^2: What do people want from DB abstraction modules?
by jZed (Prior) on Jan 05, 2006 at 18:40 UTC
    In DBI, to get a row of data using a primary key for the bind variable, I need to create a statement handler, bind a parameter, execute, and return some structure. 4 lines of code
    Or one line: my @row = $dbh->selectrow_array($sql,{},$key);
    Cross DB support. Ever need to join two tables aross do different databases?
    DBD::AnyData can make that pretty simple for some queries:
    my $data = $dbh->selectrow_array( "SELECT $cols FROM IMPORT(?) JOIN IMPORT(?) ON(...", {}, $postgresql_sth, $mysql_sth );
      Or one line: my @row = $dbh->selectrow_array($sql,{},$key);
      Never noticed that before on the $dbh object. But more of those types of things are useful.
      my $data = $dbh->selectrow_array( "SELECT $cols FROM IMPORT(?) JOIN IMPORT(?) ON(...", {}, $postgresql_sth, $mysql_sth );
      I kinda like that. The funky IMPORT(?) doesn't seem standard, but something more "SELECT * FROM $dbh1 A, $dbh2 B where b.something = a.something" seems more natural. In some databases, I frequently do, "
      SELECT * FROM ( SELECT * FROM A WHERE A.something = 1 ) AA, ( SELECT * FROM B WHERE B.something = 2) BB WHERE AA.bob = BB.bob
      nitpick pick pick :)

      ----
      Give me strength for today.. I will not talk it away..
      Just for a moment.. It will burn through the clouds.. and shine down on me.

Re^2: What do people want from DB abstraction modules?
by techcode (Hermit) on Jan 05, 2006 at 17:59 UTC

    Cross DB support. Ever need to join two tables aross do different databases? It's the pits.
    Could you describe what do you mean? My modules does support work with several databases (different dbh's) at the same time - and should support multiple DB types (MySql, SQLite, Postgree ...). But I have no idea how to possibly join two tables in different DB's (with JOIN or say sub query).

    Simple things should be made simple. In DBI, to get a row of data using a primary key for the bind variable, I need to create a statement handler, bind a parameter, execute, and return some structure. 4 lines of code I use everywhere, but I wind up keeping DBI libs so that I don't have to do that all the time. Pains me to see people write those 4 lines everywhere.
    I actualy never used bind variables. I don't like them for some reason - and find it more convenient to place everything inside hashes. Guess it's because most modules accept/give hashes in one way or another.

    But you could do something like:
    my ($var1, $var2, $var3) = $DB->execute('SELECT field1, field2, field3 + FROM whatever')->fetchrow_array();
    Or maybe even something like following.
    my ($var1, $var2, $var3) = $DB->do('SELECT field1, field2, field3 FROM + whatever WHERE id = 1');

    Requesting data also always requires SQL. Wouldn't it be nice if I can use a set of the ANSI SQL language and have it translated to my DB's version of SQL. I'm tired of converting sql statements from one db's SQL into another. Don't even start me on openning a connection for rollback-able transactions.
    Oh well this isn't perfect world. IMHO that's the job of RDBMS in the first place - they should stick to standards. Only thing I can do is support functions that will be there for as many DB types.

    Consistency in the low-level operations. I don't trust people's driver level libraries to be remotely consistent. I want to setup, breakdown and have queries adhere to timeouts in a consistent fashion. Setting up evals with alarms is something people should not have to think about.
    I'm not sure I follow you on this?


    Have you tried freelancing? Check out Scriptlance - I work there.
      Could you describe what do you mean? My modules does support work with several databases (different dbh's) at the same time - and should support multiple DB types (MySql, SQLite, Postgree ...). But I have no idea how to possibly join two tables in different DB's (with JOIN or say sub query).
      Sometimes one needs to work w/ two different database handles. I can get the data from both, and know that two columns, one from either database, can be used in a join. Only problem is, db1 and db2 know nothing about each other. Hell, they may not even be the same DB. I know the solution to the problem. Get all the data from db1 I need, get all the data from db2 I need. Iterate on one, joining results from the other using this one column.

      It doesn't come up often, but it does come up. The module would need to be fed two queries, and two joining column numbers to create a new set of results.

      Oh well this isn't perfect world. IMHO that's the job of RDBMS in the first place - they should stick to standards. Only thing I can do is support functions that will be there for as many DB types.
      They should but they don't. It doesn't make the problem go away.
      Consistency in the low-level operations. I don't trust people's driver level libraries to be remotely consistent. I want to setup, breakdown and have queries adhere to timeouts in a consistent fashion. Setting up evals with alarms is something people should not have to think about.
      The only operations I can think of that are common on DBI across multiple DB types is the username and password being setup. That's fine. I have no way of saying, "if a query being executed takes too long, abort and close the statement". DBI handles a lot of this, but not everything. There's an extra few metres left to cover.

      ----
      Give me strength for today.. I will not talk it away..
      Just for a moment.. It will burn through the clouds.. and shine down on me.

        I'm finding that I need to join between databases on multiple servers almost every day. In the future I may try using DBD::AnyData for that but am currently just exporting everything to Prolog and having swi-prolog do the joins for me.

        ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊

      Could you describe what do you mean? My modules does support work with several databases (different dbh's) at the same time - and should support multiple DB types (MySql, SQLite, Postgree ...). But I have no idea how to possibly join two tables in different DB's (with JOIN or say sub query).

      I think that's exactly what's being requested -- a join or sub query using multiple databases. Typically, you can rewrite a join as a subquery, or vise-versa, and databases typically handle two different styles of joins (sort-merge or correlated).

      With a sort-merge, you take the results from each table, sort them on the fields that it's getting joined by, and then work your way through the lists, finding entries in each list that have those fields in common.

      With a correlated query, you get the results from one table, then for each instance of the fields that you're trying to merge on, you query the other table.

      Depending on the size of the two tables being joined, one or the other may be more efficient. (and of course, with a correlated query, if it's an equijoin, you can work from either table first.) Database tuning involves storing histograms of the fields that you're going to be joining on, so you have a clue which way is going to be the most efficient.

      Oh -- and for those using Oracle on both ends -- look into what they call 'database links'.