I am currently looking at revamping a project I did about 2 years ago. At that time I put together a website that lets people enter information about what they are doing on any given day (this was a corporate project). When the Perl program assembled the record, it looked something like, "date,user,comments", with each record on a separate line. When I wanted to parse the information, I had the Perl script open up the text file that these records were stored in, shove them into memory as a hash, and pick out which ones I wanted.

It was simple, quick and easy to develop. Times have changed (for me) and I am looking for a more efficient/elegant solution, though. There are a couple of constraints: 1. No database, and 2. I cannot gaurantee the order of the records at this time.

As it stands, I sort the records by the date in memory. A user has the ability to change dates on existing records or enter records for past/future times. This means that I never know where a record in this text file might be, since a record may be appended to the file with a date drastically different then the surrounding records. So I use a linear search routine and sort after the fact.

For some reason this just offends my sensibilities.

The project is not slow (even after two years worth of data), it is not outdated, and is very useful. But I am still bothered by the way I am storing the data and how I am searching through it. There must be a better way! So my question really is this:

What is the most efficient/elegant way of storing data in a text file so it can be easily and quickly queried without setting up a full-fledged database?

Thank you all for your help!

In reply to Better Way of Storing Data in a Text File? by jupe

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.