Data storage for a flexible note-taking program?

SuperCruncher has asked for the wisdom of the Perl Monks concerning the following question:

I'm terrible when it comes to writing down notes. I write notes on any bits of scrap paper and I can find, and then of course they got lost and I lose important information. I've decided that enough is enough, and I now want to make a program that offers the flexibility of writing notes on paper, but with more advantages and none of the disadvantages. I find existing note taking programs too restrictive, so I've decided to make my own.

My main problem is finding a suitable data storage system. Before I mention my criteria/requirements, I should mention that one of the main design goals of the program is that it should be built in a 'core' with multiple interfaces on top (e.g. wxPerl, web-based or terminal). Data will be stored locally but it must be possible to sync to an external source (probably just an FTP server somewhere, but maybe WebDAV if I can figure it out).

As the data will be mostly unstructured (as users will type in the note, a few words to describe it, optionally a title, optionally a category and optionally a date associated with the note), I'm having trouble thinking of a suitable means of data storage. Each of the 'usual suspects' doesn't quite seem suitable:

Database: too awkward, too difficult to install for users. Makes syncing more difficult.
Serialised data structures (either with Data::Dumper or YAML): easy to 'parse', but relies too heavily on storing all items in memory.
XML using XML::Parser or LibXML: argh... just not sure if I can be bothered.

Any advice would be appreciated.

Comment on Data storage for a flexible note-taking program?

Replies are listed 'Best First'.
Re: Data storage for a flexible note-taking program? by adrianh (Chancellor) on Nov 22, 2003 at 14:53 UTC
My main problem is finding a suitable data storage system Nahh ;-) Your main problem is figuring out what your note taking application should do. Once you've done that picking a storage mechanism should be obvious. My advice would be to start with plain text files (and you'll be surprised how far you can go with plain text files :-). Keep a nice clear API between your application logic and your data store. As soon as you find you need something more sophisticated than plain text files use it. Swapping in a new persistence layer should be simple. Let you application's requirements drive what the data store should be, not the other way around. Also, have you considered using a wiki like CGI::Kwiki as your note taking program? Various wiki's have been doing the job for me for a year or so now - might be worth considering.	[reply]
Re: Data storage for a flexible note-taking program? by duff (Parson) on Nov 22, 2003 at 16:31 UTC
...and optionally a date associated with the note Were it me, I would timestamp every note rather than making it optional. People are always writing stuff down and forgetting where the note is, but I bet you they can remember the ball-park of when they wrote it. So, you're giving them a comon place to put all of their notes, but it sounds like the organization scheme is completely optional. Timestamping each note at least gives them an organization that they are familiar with and can use in lieu of creating their own. Serialised data structures (either with Data::Dumper or YAML): easy to 'parse', but relies too heavily on storing all items in memory. Who says you need to store all of the items in memory? Who even says that you even need the entire contents of a single record in memory? duff	[reply]
Re: Data storage for a flexible note-taking program? by Aristotle (Chancellor) on Nov 22, 2003 at 13:54 UTC
How about a tied hash? Particularly in combination with MLDBM and Storable? Makeshifts last the longest.	[reply]
Re: Data storage for a flexible note-taking program? by qq (Hermit) on Nov 22, 2003 at 23:21 UTC
Database: too awkward, too difficult to install for users. Makes syncing more difficult. DBD::SQLite will cure the installation problem. qq	[reply]
Re: Data storage for a flexible note-taking program? by ysth (Canon) on Nov 23, 2003 at 02:23 UTC
The answer probably depends on what out you mean by sync to an external source. Would this be a copy of the whole database or some kind of record-by-record update based on title, date, etc.? For purely local use, I would just make a CSV file, and be able to do searching from the shell prompt with grep.	[reply]