Re^2: Much slower DBI on RHEL6

And now that my brains are working again, a third option that is quicker than all of those on Postgres, but YMMV:

select li_i.id, description from list_index li_i
join (
    select id, max(rpt_key) as rpt_key from list_index group by id
) li_o
on li_i.id = li_o.id and li_i.rpt_key = li_o.rpt_key;
[download]

I assume here that the pair (id, rpt_key) is unique, but it should not hurt even if it isn't.

Comment on Re^2: Much slower DBI on RHEL6 Select or Download Code

Replies are listed 'Best First'.
Re^3: Much slower DBI on RHEL6 by MPM (Novice) on Feb 07, 2014 at 11:32 UTC
Even though I have things working in a sufficient manner, I'll give this a shot as well. I agree MySQL should be able to be responsible for more of the processing. I'll report back with my findings. THanks.	[reply]
Re^4: Much slower DBI on RHEL6 by MPM (Novice) on Feb 07, 2014 at 12:44 UTC
WOW! Your suggestion seems to work great!!! Prior to your modification I found tux's suggestion was much faster on RHEL6 compared to the original code on RHEL6. On RHEL 4, tux's suggestion seemed to be just a slight bit faster then the original code on RHEL4. Adding in your query make is very fast on both platforms. Thanks so much! Now, if possible and if it is not to much to ask, can you break down what that query is actually doing? I don't have any experience with queries that complicated. Pretty much just simple selects and updates. Thanks again!	[reply]
Re^5: Much slower DBI on RHEL6 by Anonymous Monk on Feb 07, 2014 at 13:47 UTC
Your original method is two queries. You first query for the unique values and then go over every row of it and repeatedly launch a new query to fetch the related value. Fetching related values is usually called a 'join' in relational-database speak. It happens that relational databases are veeery good at joins. It's what they do for a living. What I did was that I actually changed your two queries into... well, two queries that are linked by a join. That way they will be sent to the database just once, the database figures out the result, and returns it to your program. The one query in parentheses with the `max()` is known as a subquery and it returns an intermediate resultset. It's basically a second (temporary) table. It fills the role of your LIMIT 1 query, except that it returns all the related rows rather than just one. This temporary resultset is then joined against the main table where the `id` and `rpt_key` columns match. This gives you the resultset you previously used Perl to build -- in a single query. You can take the two small queries out of the big query and run them against the database and examine the result to see what they return and how it works. If you work with relational databases, you really ought to learn SQL to a level that is above the very basic SELECTs and INSERTs. It'll require some stretching of the brain but it's worth it.	[reply] [d/l] [select]