How about making sure you create a unique key for each point, then serializing the data points (Freeze() is one way, Conway's OOP book lists several), and shoving them in a tied DBM structure?

The caveat here is making sure you know WHICH DBM your product will be using: "Bucket-size" limitations vary greatly, and with 1000 objects, if you are using (for example) the "standard" DBM (nDBM, IIRC) on Solaris 2.6, you're going to either have 65-byte objects or run out of space; other DBM implementations have much larger bucket-size limitations, and at least one is much smaller.

The big advantages of DBM are that it's easy to implement/maintain (there's even the explicit-but-deprecated dbmopen() command, if tie() is too heady for your future maintainers), data manipulation is easy (just regular hashes), and it's fairly quick (YMMV, again depending on which version of DBM you're using).

The big drawback (apart from the bucket size problem described above) is that there is no internal locking mechanism in the DBM specification; if you need locks (will multiple scripts be rewriting the same object simultaneously?), you have to implement them yourself.

That said, there are some other games you can play to push the performance even higher. One is that if you have a ramdisk, or a system that supports a shared swap//tmp filesystem (many commercial versions of Unix will overlay swap space with the /tmp filesystem -- if the machine is not swapping, /tmp is memory-mapped; if it is, swap data starts getting written into /tmp), you can put the DBM in the memory-mapped space (good idea, though, to copy the file somewhere more static every so often as a backup), so that reads/writes with the tied DBM never actually touch disk.

As far as optimizing the CGI (and serializing the DBM connections), one approach might be to have a separate daemon which babysits the DBM, and communicates back and forth with the CGI instances via named pipes. This way, you wouldn't have to worry about the overhead of the CGI actually opening the DBM each time.

Of course, you never specified what platform you were implementing this on. All of my advice with this goes for naught if you aren't on a Unix or Linux (or OS X??) system.

Spud Zeppelin * spud@spudzeppelin.com

In reply to Re: CGI and static data... by spudzeppelin
in thread CGI and static data... by dragonchild

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.