in reply to Database or Storable?

Not enough information. How many items of data? Does the information need to persist between runs of the application? How often per second do you need to access the data? Do you access the items randomly or do you need it all at once? How often does the data change? Does the process reading the data also change the data. Is all the data changed at once or are random items changed?

True laziness is hard work

Replies are listed 'Best First'.
Re^2: Database or Storable?
by Anonymous Monk on Mar 09, 2009 at 09:17 UTC

    Thanks, and my apology for not offering sufficient details.

    The questions are stored in the database (MySQL). The total statistics need to be accessed every time the web page is accessed, because they are displayed e.g. (Maths -> 3000 questions, English -> 2050 questions, etc). The total statistics don't change frequently, only when new questions are added or when existing ones are deleted (but that doesn't happen so often). Speed of access is important - the faster, the better.

    I've saved the total statistics in Perl Storable. I'm guessing it's more efficient to get these values from the stored hash reference than querying the database. But I'm not exactly sure.

[reply]

      The conventional wisdom is: profile it or bench mark it. Maybe you can do that with an off line version of the data and a test script?

      The gut feeling is: the database is probably faster if it's a simple query. Maybe it would be worth adding a table to cache the stats that are referenced often, although without knowing the current structure of your database and how the totals are obtained it's hard to tell.

      True laziness is hard work
[reply]

      How much of the total dataset is required by any given access to the web page?

      If, for example, only the Math questions are required by any given load, consider splitting the Storable datset into 2 and only loading the required set once you know which is required. You'll need to do a directory lookup anyway, and if you can defer that lookup until you've acquired the information telling you which half you need, the page will load more quickly and you'll only need to load half the data when you do.

      If there is a natural split that reduces the amount of data further, try that also.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        The questions themselves are stored in the database, and are only accessed when the user selects a practice. The Storable came about because I was thinking it could be used to store the total statistics - not holding any questions. I could get the totals from the database, using COUNT(*) for instance, but I was wondering whether it might be more efficient to get these values from a stored hash reference, instead of querying the database.
[reply]

In Section Seekers of Perl Wisdom