global has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

just starting down the road with PERL and trying to work with databases. I am using Postgres and Cisco Netflow data. I am creating a table for the day eg 20030219 and would then like to just add my data to the table but for each flow file I would like to create the table if it isn't already there BUT ignore the warning message that is created if it is already there.

The $sth->execute or die statement needs to basically handle ignore the situation if the table is already created or die if there is any other problem. Unless there is a better way in Postgres to handle this. Thanks, Hugh

  • Comment on Ignore Relation already exists error from DBI

Replies are listed 'Best First'.
Re: Ignore Relation already exists error from DBI
by Tomte (Priest) on Feb 19, 2003 at 12:17 UTC

    Two remarks:

    • As I don't know your application, take this with a grain of salt, but: If I hear a thing like: "creating a table for the day eg 20030219 and" an alarm bell starts to ring in my head and it tries to say: Rethink your design! This doesn't sound right.
    • As far as a quick glance (off site) got me, you will have to check the dbi-error before dying, as IF NOT EXISTS or the like is not mentioned.
      Code similar to
      $sth->execute(); # $sth is the statement-handle if ($dbh->errstr && $dbh->errstr =~ /Relation\s[a-zA-Z_-']+\salready\s +exists/) { # $dbh is the db-handle die ... }
      should get you started. but I strongly suggest to rethink your design! instead.

    regards,
    tomte


Re: Ignore Relation already exists error from DBI
by robartes (Priest) on Feb 19, 2003 at 12:24 UTC
    You want to check the value of $DBI::errstr. Here's an example:
    $sth->execute or do_error($DBI::errstr); sub do_error { my $error=shift; return if $error =~ /Relation already exists/; # or whatever the ac +tual error is die "A serious problem occured. According to the rumours: $error.\n" +; }
    Note that the key here is not to die on a return value of undef from the execute, but to execute a subroutine that does some extra checking (which you can make as elaborate as you like).

    Update: the above answers your immediate question, but I agree with tomte that your database design probably needs some work.

    CU
    Robartes-

      Thanks guys for the ideas I will just have to get the regular expression correct.

      I am dealing with Netflow data which records flows through a router. I am just grouping the flow data from each day into a seperate table until it is a week old and then I will aggregate the data some more.

      Does this still sound like a bad design?

        Does this still sound like a bad design?

        To my ears, it does :-)
        The date of the day makes for a perfect column, if you think about it.

        regards,
        tomte


        I am just grouping the flow data from each day into a seperate table until it is a week old and then I will aggregate the data some more. ... Does this still sound like a bad design?

        Yes. By keeping data in separate tables, you're making more work for yourself. Databases are good at aggregating information, and it's easiest if all of the data is in a single table.

        If I was doing this, I'd keep add a (non-key) field to hold the date, and would keep all data in a single file. Then, I could do something like

        SELECT sum(somefield) FROM table WHERE datefield BETWEEN ? AND ?
        to do aggregate calculations. Or
        SELECT datefield, sum(somefield) FROM table WHERE datafield BETWEEN ? AND ? GROUP BY datefield
        if I needed the aggregates to apply to distinct dates.

        Flushing old data is easy.

        DELETE from TABLE WHERE datafield < ?
        These are all static queries. By keeping all data in a single table, I don't have to reconstruct new queries to plug in a dynamic table name. And I'm letting the database do the aggregate calculations, rather than pulling all of the data over the wire to the client and doing it there.

        If your particular calculations are more complicated, and can't be done in SQL, you may still be able to do part of the aggregation on the server side, reducing the amount of data you have to pull across the wire into the client.

Re: Ignore Relation already exists error from DBI
by diotalevi (Canon) on Feb 19, 2003 at 14:12 UTC
Re: Ignore Relation already exists error from DBI
by paulbort (Hermit) on Feb 19, 2003 at 16:02 UTC
    Since you're using PostgreSQL, you can ask it directly if a table exists, with something like this:
    SELECT relhasindex, relkind, relchecks, reltriggers, relhasrules FROM pg_class WHERE relname='day_20030218';
    I also have to suggest that separate tables for each day is a poor use of the power of a database unless you are running into per-table capacity issues (unlikely with PostgreSQL.) There's a good set of pages from the University of Texas about database design that I point people to all the time.

    --
    Spring: Forces, Coiled Again!
Re: Ignore Relation already exists error from DBI
by Jonathan (Curate) on Feb 20, 2003 at 11:27 UTC
    If you just want to ignore the table creation error why not perform the sql in a block with local overrides for DBI's RaiseError and PrintError methods? You will, of course, have to then explicitly test your SQL statements outcome but you will have complete control over what is fatal and what you're happy to ignore. perldoc DBI for more information