IMHO you are digging yourself a hole. An object where you want one chunk of properties in one circumstance and another set of properties in another smells like two or more objects to me. You sound like you are trying to optimise somthing that does not even work yet. Chances are the bottlenecks won't be where you suspect.

You don't seem to understand what takes time and what does not. When you retrieve data from a DB it takes almost exactly the same time to pull one byte as it does to pull lots of bytes. The reason is that a disk will deliver something like 50MB per second and the connect/parse/bind/execute/seek/disconnect overhead is the same (almost) no matter how much data you pull. The actual data delivery time is ~ 10% or less of the total transaction time.

If you ignore the simple fact that it is far easier/faster for a DB to pull a chunk of data (say a serialised object) than it is for it to pull a whole lot of bits from that 'object', your approach will be be slower simply because the best case is you shave a few % off a single transaction (it is arguable it will not take *longer* due to increased query complexity) but a second transaction at any stage will add 95% + which totally wipes out any gain.

I would suggest you rethink your approach. KISS and just pull all the data you need in one pass. If when you have it running you find a bottleneck then look at optimising it.

Storable and FreezeThaw are two good serilisation solutions. Typically you don't want to store serialised object in a DB as you loose most of the value of using a RDBMS, but if you have good reasons to do it these modules can help.

cheers

tachyon

In reply to Re: algorithm help for determining efficient db data retrieval by tachyon
in thread algorithm help for determining efficient db data retrieval by AidanLee

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.