Re^4: Much slower DBI on RHEL6

WOW! Your suggestion seems to work great!!! Prior to your modification I found tux's suggestion was much faster on RHEL6 compared to the original code on RHEL6. On RHEL 4, tux's suggestion seemed to be just a slight bit faster then the original code on RHEL4. Adding in your query make is very fast on both platforms. Thanks so much! Now, if possible and if it is not to much to ask, can you break down what that query is actually doing? I don't have any experience with queries that complicated. Pretty much just simple selects and updates. Thanks again!

Comment on Re^4: Much slower DBI on RHEL6

Replies are listed 'Best First'.
Re^5: Much slower DBI on RHEL6 by Anonymous Monk on Feb 07, 2014 at 13:47 UTC
Your original method is two queries. You first query for the unique values and then go over every row of it and repeatedly launch a new query to fetch the related value. Fetching related values is usually called a 'join' in relational-database speak. It happens that relational databases are veeery good at joins. It's what they do for a living. What I did was that I actually changed your two queries into... well, two queries that are linked by a join. That way they will be sent to the database just once, the database figures out the result, and returns it to your program. The one query in parentheses with the `max()` is known as a subquery and it returns an intermediate resultset. It's basically a second (temporary) table. It fills the role of your LIMIT 1 query, except that it returns all the related rows rather than just one. This temporary resultset is then joined against the main table where the `id` and `rpt_key` columns match. This gives you the resultset you previously used Perl to build -- in a single query. You can take the two small queries out of the big query and run them against the database and examine the result to see what they return and how it works. If you work with relational databases, you really ought to learn SQL to a level that is above the very basic SELECTs and INSERTs. It'll require some stretching of the brain but it's worth it.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^5: Much slower DBI on RHEL6
by Anonymous Monk on Feb 07, 2014 at 13:47 UTC

It happens that relational databases are veeery good at joins. It's what they do for a living. What I did was that I actually changed your two queries into... well, two queries that are linked by a join. That way they will be sent to the database just once, the database figures out the result, and returns it to your program.

The one query in parentheses with the max() is known as a subquery and it returns an intermediate resultset. It's basically a second (temporary) table. It fills the role of your LIMIT 1 query, except that it returns all the related rows rather than just one.

This temporary resultset is then joined against the main table where the id and rpt_key columns match. This gives you the resultset you previously used Perl to build -- in a single query.

You can take the two small queries out of the big query and run them against the database and examine the result to see what they return and how it works.

If you work with relational databases, you really ought to learn SQL to a level that is above the very basic SELECTs and INSERTs. It'll require some stretching of the brain but it's worth it.

[reply]
[d/l]
[select]