in reply to Eliminating Dynamic SQL Generation with Decision Matrices for Tighter Large Scale Applications

This is an interesting post. I agree with the second and third issues you bring up (tuning and task division), although I think that in many applications the third issue isn't sizable since it's the same developer with a different hat on :-) But you're focusing on large-scale applications so I understand.

Regarding the first issue: I'm curious if you've measured the time needed to build the queries using the methods you describe here, or others more complicated.

I like the general concept behind generating SQL statements beforehand, particularly for queries that rely on multiple tables, because it's easier to debug and to test. However, this seems to conflict with the desire of many users to perform ad-hoc queries, even if constrained by a simple web form. In cases like this, do you think it's a good idea to generate all the queries beforehand? This would seem to be a query-management nightmare since you have to account for every combination of criteria.

This might lead to a hybrid system where often-used queries are generated beforehand but ad-hoc queries are generated (partly at least) at run/request-time. I'm thinking of including something like this in SPOPS as it would add a new wrinkle to managing object relationships.

Chris
M-x auto-bs-mode

  • Comment on Re: Eliminating Dynamic SQL Generation with Decision Matrices for Tighter Large Scale Applications

Replies are listed 'Best First'.
Re: Re: Eliminating Dynamic SQL Generation with Decision Matrices for Tighter Large Scale Applications
by mpeppler (Vicar) on Jan 17, 2002 at 02:43 UTC
    ...generating SQL statements beforehand... seems to conflict with the desire of many users to perform ad-hoc queries, even if constrained by a simple web form.

    It all depends on the size of the database, the type of queries, and the load that is placed on the database (i.e. the number of parallel instances of such ad-hoc queries).

    Personally I've worked on several large systems (for example the now defunct eCircles.com site) where all access to the database is via pre-defined stored procedures (which is equivalent to pre-defined queries, really). Obviously it requires some thought to define the set of operations that users should be allowed to perform, but it has the huge advantage of separating the SQL from the application. If a query performs badly it can be tuned by the DBA (or the SQL developper) without touching the perl code (as long as the input and output stays the same, of course).
    The queries can also be analyzed in isolation to ensure decent performance.

    It is, IMHO, a must.

    Michael