If you move to a real DB (or even with a slight modification to the current format) there is a useful trick - add an update time field. Unix epoch time is fine. Thus your master database looks like: 

data_field1 data_field2 ... update_time

[download]

With the update time to hand it is a simple matter to select all the records with an update time > some_value and thus generate a record set than encapsulates all the latest changes, valid from any fixed time point.

It looks to me as though your data storage format is going to give you issues but this depends on what you are actually doing. There are essentially 3 types of data changes you may need to deal with:

  1. INSERTS - adding a brand new entry
  2. DELETIONS - removing dross
  3. UPDATES - ie what if username@domain.com changes their username? You either need to UPDATE their record or delete the old record and insert the new one.

cheers

tachyon

In reply to Re: Perl version of 'diff'? by tachyon
in thread Perl version of 'diff'? by ninja_byte

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.