in reply to A profiling surprise ...

There is an art to making DB-bound applications run efficiently. The short of it is "Index, index, index".

Now, another solution would be to construct your data such that the hit pileups can be detected as part of a SQL query. This isn't as hard as it sounds. Take a look at the MySQL function COALESCE().

My criteria for good software:
  1. Does it work?
  2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Replies are listed 'Best First'.
Re^2: A profiling surprise ...
by chexmix (Hermit) on May 23, 2008 at 20:54 UTC
    Hmm ... walking to and from the barbershop about an hour ago, I was indeed thinking that "perhaps something could be done data-wise to minimize the number of database calls."

    I'll take a look at that MySQL feature ... am stuck with Sybase here, I'm afraid.

    At least I got a haircut. :D

[reply]
      The key isn't to reduce the number of database calls. It's to reduce the overall impact of those calls. Let's say that it takes 5 seconds to calculate one of these things, of which 4 seconds is finding and retrieving data. Now, let's say you could rewrite how you retrieve the data to take 3 seconds to retrieve and only .5 seconds to work with. So, 5 seconds drops to 3.5 seconds. Now, let's say that you could do 10 of these in the same 3.5 seconds.

      This isn't just idle speculation. Over the years, I've done 99% and even 99.99% improvements in DB-bound programs. A lot of it has to do with how you store the data. Just because you collect it in one form doesn't mean you search it in that form.

      My criteria for good software:
      1. Does it work?
      2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
[reply]

In Section Seekers of Perl Wisdom