in reply to Advice of picking the language for my job, please

In my experience I would say that you should NOT go with PERL to do any of your DB work. I am doing exactly what you want to do (Extracting XML and building XML files and inserting them in to the DB). PERL has an especially difficult time interfacing with PostGRES and there are currently some bugs and efficiency issues in the existing CPAN modules that might allow you to do this.

My suggestion is that you use C to make a database abstraction layer, where all your logic and database calls are made through C. All your server-side programming can be done through PERL (because perl is excellent when it comes to working with verbose data) making function calls to your C API. You should never have to write any SQL in the perl (just DB function calls defined in your C).

This has proven to be a very efficient and ultimately time-saving process. It has also helped in debugging and optimizing our DB performance.
  • Comment on Re: Advice of picking the language for my job, please

Replies are listed 'Best First'.
Re^2: Advice of picking the language for my job, please
by ctilmes (Vicar) on Feb 04, 2005 at 17:40 UTC
    I might suggest that the fact that you made a database abstraction layer at all led to your benefits rather than the choice of C for that layer.

    I suspect you would have realized similar benefits had you written the abstraction layer in Perl.

      Like I said before, we have tried this before and Perl has some serious efficiency issues with respect to PostGres. When you start getting large data sets back it will simply crash. Also since we noticed that the current PostGres CPAN modules were buggy we decided that instead of re-writing them in PERL we would just go ahead and do it in C. Of course it all boils down to design preferences and what the Developers are comfortable with.
        I'm sorry that you've been downvoted for your comments. Another poster has reported similar experiences with perl/PostGres and no-one has posted about their first hand success with the combination. I appreciate your impartial comments, and I'm a little ashamed that our order disaproves.

        Ardemus - "This, right now, is our lives."
Re^2: Advice of picking the language for my job, please
by wazoox (Prior) on Feb 05, 2005 at 08:06 UTC
    Perl exists as a built-in tool to postgreSQL, it's called pl/perl. See PL/Perl documentation from postgres website
    It's simply that using DBI to interface postgres to perl is NOT the best way; but perl integrates extremely tightly and efficiently to postgres.
      I've been thinking of using Perl with PostgreSQL recently for a proejct. Can you tell me why DBD::Pg is any less efficient than DBD::mysql?
        I wouldn't say that. Actually postgreSQL is much more powerful than MySQL, so you'll more often hit the limits of DBI/DBD using postgres than Mysql...
        This issue recently came up on the mod_perl list. I brought the issue to the PostgreSQL irc channel and apparently libpq doesn't return rows individually as you might expect, instead loading the data into memory and then returning them one by one on a fetch basis IIRC. See this thread for a summary of the issue and a suggested solution using cursors in conjunction with DBI to handle retrievals of many rows from a PostgreSQL database.