You're not the only one who deals with huge tables. I haven't done the billion+ in a given table, but I have done queries against million+ JOIN million+ JOIN million+ for reports that had to return in under 5 seconds, including rendering to the user.

Unless those were outer joins, the number is likely much less than 1E18 (I'm assuming MySQL's optimizer is smart enough to minimize join processing). Did they include any range joins on non-index columns ?

That's funny ... I was doing data mining with MySQL. (That's those 1E18-row queries I was talking about before.) Access is limited, but it was never meant to be a RDBMS - that's what SQL*Server is for. MySQL may have been limited in the past, but it is a fully-functional RDBMS that supports ACID transactions and complete failover.

I'll take your word for it, tho my review of their SQL syntax leads me to believe they still have some way to go. (I've obviously stepped into the MySQL vs. the World hornet's nest, I should know by now not to bring it up..)

I'm curious - what's Oracle's version of LIMIT again? What about Oracle's restrictions on full-text searches? Or, how about Oracle's inability to turn ACID off when I, the developer, know I will never want it? Or, was it the overly-complicated clustering technology that Oracle itself agrees has a single failure point? Please elaborate ...

Hmm, since I never mentioned Oracle in my post, I'm not certain what question I'm being asked to elaborate on?

It's also a source filter. Source filters have a few drawbacks:

* Slower than a non-filter - they have their own parsing section which is run in Perl

A relative factor. If the application is just running "SELECT NOW()", you're quite right. If however, you're doing a large scale complex query, or multiple queries, the net impact of the source filtering is likely much less than the actual database processing time. And, of course, if run under a persistent environment, the source filtering happens during the load, not runtime. Since SQL::Preproc just emits DBI calls, runtime performance should be comparable to a DBI app.

* Can make mistakes - Only perl can parse Perl, which includes non-Perl stuff

Hmmm, last time I checked, perl can make mistakes too (thats part of the reason the release numbers have that 3rd digit). For that matter, I've heard that perl programmers occasionally make mistakes. So, based on your logic, I guess we all should stop programming in perl...or perhaps altogether.

* May not work in a persistent environment correctly - this includes mod_perl

Can't answer that one, except to say I don't see anything in SQL::Preproc that persists anything the application can't override.

SQL::Preproc can make some things quicker/simpler to implement. Its intended as a "simple things should be easy, and hard things should be possible" solution for developers that are comfortable with SQL.

Personally, I drink the "Separation-Of-Concerns" KoolAid™ and keep my Perl and SQL separate. This means using either Class::DBI (if I want an OO abstraction) or DBI (if I want performance).

A matter of preferences, then. Some folks prefer a little mustard on their hot dogs, and some of us prefer a little hot dog with our musard. I guess the point I was originally trying to make was that I've seen many instances where separation/encapsulation led to throwing an object wrapper around a cursor on a large table, and processing every returned row on the client, in perl (or Java or C++, or whatever), and ignoring the capability of the underlying DBMS and/or its SQL syntax (MySQL included!). Once those little wrappers get written - and esp. if they're written to a least common denominator SQL -, they tend to take on a life of their own, and opportunities for exploiting powerful DBMS features get ignored or shut out.


In reply to Re^3: Should it be database independent? by kimanaw
in thread Should it be database independent? by szabgab

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.