mring has asked for the wisdom of the Perl Monks concerning the following question:

Hello mighty Monks,

I am familiar with vanilla usage of the DBI but today I need to do something more complex. I need to join tables in totally different databases.

I have a firebird 1.5 DB and an oracle 9i DB. Firebird has usage tables I want to report on and Oracle has the granular user/location detail I need. There is a common UID column coexisting in both DBs to join with. For various reasons the data just has to stay where it is, I can’t move the oracle data into firebird or vice-a-versa.

I could build a meta-database and pull all the data from both sources into it on a scheduled basis, then report on that. I would prefer not to have to maintain an entirely separate database. Google pulled up DBI-Link which does what I want as far as creating pseudo tables from external data sources, but it appears to only work inside of postgreSQL, which takes me back to the separate database issue.

What I would like to do is build the join in code using Perl DBI. Is this possible or am I barking up the wrong tree?

TIA Mike

Replies are listed 'Best First'.
Re: DBI sanity check
by jZed (Prior) on Nov 10, 2005 at 16:32 UTC
    One way is to use any of the SQL::Statement DBDS - AnyData, CSV, DBM, etc. with the new IMPORT() function which allows heterogeneous joins across DBMS types. For example:
    my $csv_dbh = DBI->connect( 'dbi:CSV...); my $ora_dbh = DBI->connect( 'dbi:Oracle...); my $fir_dbh = DBI->connect( 'dbi:Firebird...); my $ora_sth = $ora_dbh->prepare(...); my $fir_sth = $fir_dbh->prepare(...); my $csv_sth = $csv_dbh->prepare(" SELECT $cols FROM IMPORT(?) AS T1 NATURAL JOIN IMPORT(?) AS T2 ... "); $csv_sth->execute( $fir_sth, $ora_sth );
    The syntax for the join must follow the SQL::Statement rules, while the two statement handles follow the syntax rules of the respective DBMSs they query.
      Thank you both for your answers, I will dig in and start reading up on both options.
Re: DBI sanity check
by jk2addict (Chaplain) on Nov 10, 2005 at 16:31 UTC

    I don't believe a true cross-server join is possible. Depending on your reporting requirements, or how you are going to interact with the data, you could get at the data with a lot less coding work by using something like Class::DBI. You would essential create two classes, on for each DB, then declare the has_a/many relationships between the two. The underlying code would take care of joins the data when accessing the relationships.

    From the CDBI pod:

    NOTE: The two classes in a relationship do not have to be in the same database, on the same machine, or even in the same type of database! It is quite acceptable for a table in a MySQL database to be connected to a different table in an Oracle database, and for cascading delete etc to work across these. This should assist greatly if you need to migrate a database gradually.

Re: DBI sanity check
by radiantmatrix (Parson) on Nov 10, 2005 at 17:59 UTC

    All of the above suggestions are excellent. Let me give you the most basic form of a generic solution. This is probably not the best solution, but hopefully seeing some code will whet your mind a bit. I've left out gobs of error handling, since you said you are already familiar with the basics of DBI.

    use strict; use warnings; use DBI; #replace these with the correct connect strings, of course my $usage_db = DBI->connect('dbi:ODBC:usage','',''); my $detail_db = DBI->connect('dbi:ODBC:detail','',''); my $usage_st = $usage_db->prepare(q' SELECT uid,description,err_code FROM errors WHERE err_code = ? '); my $detail_st = $detail_db->prepare(q' SELECT * FROM error_details WHERE uid = ? '); #- CORE -# my $fatals = get_general('FATAL'); # get all fatal errors; foreach my $record (@$fatals) { my $uid = shift @$record; my $detail = get_detail($uid); #output result and its detail print join('||',$uid, @$record, @$detail), "\n"; } #- END CORE -# # this sub finds all records with a given err_code sub get_general { my $code = shift; $usage_st->execute($code); my $res = $sth->fetchall_arrayref; return $res; } # this sub gets the details for a given uid sub get_detail { my $uid = shift; $detail_st->execute($uid); my $res = $sth->fetchall_arrayref; return $res; }
    <-radiant.matrix->
    A collection of thoughts and links from the minds of geeks
    The Code that can be seen is not the true Code
    "In any sufficiently large group of people, most are idiots" - Kaa's Law
Re: DBI sanity check
by terce (Friar) on Nov 10, 2005 at 16:42 UTC
    It's not clear from your description exactly what role Perl needs to play in your problem. Are you using it to generate the output of your report, or are you looking at Perl purely as a tool to make the join?

    Assuming that the end-result of the join query needs to be available to Perl, a couple of possibilities spring to mind.

    The first, although it doesn't meet your requirement that another database not be used, is that you use a SQLite database (created and accessed using DBD::SQLite) to create a temporary store for the source data from your report. This would leave you with a single SQLite file in your filesystem which you could discard when your reporting process had completed, then regenerate the next day. You get the benefit of a (more or less) SQL-92 compliant RDBMS, without any of the headache of management.
    SQLite is fast, even with large datasets - it's limitations are mostly when supporting multi-user environments, which shouldn't be an issue for offline reporting.

    The second (non-Perl) option would be that you explore making the Oracle database visible from Firebird, or vice-versa, using whatever remote datasource tools are available. I don't know Firebird, but I'm certain Oracle has tools to enable you to connect to and use other databases in Oracle queries. Other, more Oracle-literate Monks may have suggestions here.
      Terce,

      Thanks for the reply. To clairify, I've already writen a few reporting scripts based on DBI and GD::Graph, I will be modifying them for this task, so yes the end-result needs to be perl accesable.

      I've read about SQLite in sysadmin journal, it sounds cool, but I'm going to try the perl methods posted here first.