Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

SQL question: Insertion with FK constraint

by lihao (Monk)
on Jun 26, 2008 at 14:45 UTC ( [id://694170]=perlquestion: print w/replies, xml ) Need Help??

lihao has asked for the wisdom of the Perl Monks concerning the following question:

Hope it's not over OT. I have several MySQL tables connected/constraint by Foreign Keys, for example:
CREATE TABLE `table1` ( ...... `property_id` int(8) REFERENCES `table2` (id) ON UPDATE CASCAD +E ON DELETE NO ACTION, ...... ) ENGINE=InnoDB; CREATE TABLE `table2` ( `id` int(8) PRIMARY KEY AUTO_INCREMENT NOT NULL, `property` varchar(50) ) ENGINE=InnoDB;

When updating `table1`, I need to check `table2` and see if the given `property` exists, if it exists, return its `id` to table1.property_id; otherwise, insert it into table2 and return last_insert_id() to table1.property_id. Currently I am using DBIx::Simple, so I do it the following way (sample code):

my ($pid) = $db->query(<<END_SQL, $property)->list; SELECT `id` FROM `table2` WHERE `property` = ? END_SQL if (not $pid) { $db->query("INSERT INTO `table2` (`property`) VALUES (?)", $pr +operty); $pid = $db->last_insert_id(); }

Since there are many places(tables and columns) in my DB that need to be updated under the FK constraints. My question is: Are there simpler ways to handle this situation nicely without using the above code?

many thanks

lihao

Replies are listed 'Best First'.
Re: SQL question: Insertion with FK constraint
by perrin (Chancellor) on Jun 26, 2008 at 15:39 UTC
    Don't use that SQL syntax with REFERENCES right after the column definition. You need to define your foreign keys at the end of the table definition after all columns are declared or else MySQL will ignore it.
      This is completely incorrect for any version of MySQL 4+. 3.23 had this problem. It's been fixed.

      My criteria for good software:
      1. Does it work?
      2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
        From the docs for 5.0:
        The inline REFERENCES specifications where the references are defined as part of the column specification are silently ignored by InnoDB. InnoDB only accepts REFERENCES clauses when specified as part of a separate FOREIGN KEY specification.
Re: SQL question: Insertion with FK constraint
by pc88mxer (Vicar) on Jun 26, 2008 at 15:04 UTC
    I would just create a subroutine to translate a property to a property id. Here's one that also implements a cache:
    my %property_id; sub property_id { my ($db, $property) = @_; my $pid; unless (defined($pid = $property_id{$property}) { ($pid) = $db->query(..., $property)->list; unless (defined($pid)) { $db->query("INSERT ...", $property); $pid = $db->last_insert_id(); } $property_id{$property} = $pid; } return $pid; }

    When inserting a record that has a property FK, just call the subroutine for its value, and it will get created if necessary.

Re: SQL question: Insertion with FK constraint
by roboticus (Chancellor) on Jun 27, 2008 at 13:04 UTC
    lihao:

    I do this sort of thing all the time, primarily for bulk-loading data. The method I usually use is to first load the data into a holding table (no keys, so loading is fast & easy):

    create table #HOLD ( ... property varchar(50), ... record_state char(1) )

    I next check constrained columns and do some combination of:

    • (a) report new values found in the table,
    • (b) mark the records in the holding table that contain unknown values,
    • (c) generate new entries in the dictionary table, or
    • (d) delete the records containing the unknown values.
    -- (a) Report new values select distinct property from #HOLD where property not in (select property from table2) -- (b) Mark records containing those new values update #HOLD set record_state='N' where property not in (select property from table2) -- (c) Generate new dictionary items (assumes auto-incrementing ID col +umn) insert table2 (property) select distinct property from #HOLD where property not in (select property from table2) -- (d) Delete records containing an unknown value (assumes you didn't -- add new dictionary items) delete #HOLD where property not in (select property from table2)

    I can then update the main table(s) without worry:

    -- Alter any records needing update update table1 set .... property_id = T2.id .... from table1 T1 join #HOLD H on H.PrimaryKeyColumn=T1.PrimaryKeyColumn join table2 T2 on T2.property = H.property where T1.property_id is null or T1.property_id != T2.id -- Insert new records insert table1 (..., property_id, ...) select ..., T2.id as property_id, ... from #HOLD H where H.PrimaryKeyColumn not in ( select PrimaryKeyColumn from table1 )

    Finally, I can get rid of my holding table.

    ...roboticus
Re: SQL question: Insertion with FK constraint
by psini (Deacon) on Jun 26, 2008 at 14:57 UTC

    I don't use MySQL (I'm a Pg fan) so there could be some way but, if so, it is not standard SQL

    I think your approach is right (i.e. I don/t know a better way to do it) but I'm wondering about the correctness of the idea behind it: if your table2 has only the column property and it is automatically populated when you insert into table1, what is the need for a separate table?

    Rule One: "Do not act incautiously when confronting a little bald wrinkly smiling man."

      psini, thank you very much for your reply.. this is actually an example, so there are probably more fields in table2.:)

      In fact, even with just two fields, using a separate table sometimes save me lot of disk spaces and time, i.e. if 'property' varchar(20) becomes varchar(250) or text. In fact, I just refactored one of my big table from 4GB to 2.2GB yesterday by such process(normalization). the new table took 4 hours instead of 2 days to build a huge Unique key constraint. :-)

        Ah, OK. If data grouped in table2 is significantly complex I'd expect a little performance gain. But with an efficient DBMS I'd believe that the overhead added by joining the tables should lose the gain from memory saving.

        BTW, 4 days to build a unique index seems to me really too much! I have a table with about 10 million rows (but only 6 columns, I've to admit) that build its primary key (on two columns) in about an hour. And last time I had to restore it from a backup I grumbled for it was too slow... :)

        Rule One: "Do not act incautiously when confronting a little bald wrinkly smiling man."

Re: SQL question: Insertion with FK constraint
by dragonchild (Archbishop) on Jun 26, 2008 at 15:59 UTC
    The method you're using is just fine. Though, I would argue that you shouldn't be trying to insert a property into table1 without having made sure it exists in table2 first. I think you're putting the cart before the horse somewhere in your application's logic.

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://694170]
Approved by psini
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (3)
As of 2024-04-25 23:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found