Let me re-state your question: "What's a good way to efficiently load a complex but infrequently changing data structure that I want to display in my mod_perl application?"

MySQL is fast enough for most uses like this, but if you have an unusual amount of traffic or a poorly designed schema and your data does not need to be 100% up-to-date, caching is tremendously helpful. The approach you described will work just fine. I would suggest you use Cache::Cache or MLDBM::Sync instead of rolling your own filesystem database though. The biggest problem with this approach is that your data set may become too large to efficiently regenerate all of it every 5 minutes.

I used a load-on-demand approach with caching for the eToys.com product data. This avoids problems stemming from a large data set, but does result in a slower load time for the first user. Here's a description:

• Use Perl objects to represent your data. Give them a load() method that takes a unique ID.
• Inside the load method, check the local cache (I used BerkeleyDB, but you can use one of the modules I suggested above) and retrieve this object if it's there.
• If it isn't in the cache, load it from the database. Put the loaded data in the cache.

You can vary how much your cached data gets out of sync by adjusting the time-to-live for objects in the cache.

Caching is a big subject, and there are lots of tricks you can use (lazy reloading, pre-filling the cache with hot objects, etc.).

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.