I know that this is not what you want, but I wrote me a file indexer which uses DBD::SQLite or any other DBI database. Insert-or-update is built-in, but not atomic, but as SQLite is my main focus, I know that I'm the only user on it while I have the connection...

Insertion of a new item is as fast as the MD5 can be computed (if you enable MD5s), and scanning of the database for stale files (that is, files that don't exist in the real world anymore, or changed their timestamp or size) is at the cost of a stat call per file. That's not ideal, but it's hard to imagine how it could get faster and not fall prey to the complexity that File::Find has regarding file systems.

The design is very simple - a single table, containing the filename as an absolute file:// URI, and the file size, timestamp and MD5, plus arbitrary other metadata. All metadata goes into that table as columns, so once you add audio files and images, that table becomes denormalized, but that's a small price to pay for very simple queries.

The synopsis code of the module looks like this:

  use FFRIndexed;
  my $idx = FFRIndexed->new( create_if_needed => 1 );

  $idx->scan('/home/media/mp3/');
  # this will rescan all files and stat them ...

  my $files = $idx->query( { mime_type => 'audio/mpeg', artist => 'Jam
+iroquai' });
  printf "%s - %s - %s\n", $_->album, $_->title, $_->uri for @$files;

[download]

In fact, the Table::Denormalized distribution contains a complete "media files browser" application, mostly geared towards interactively browsing a collection of audio files. If anyone wants to look into this closer or even adopt modules out of it, you're very welcome!

In reply to Re: What kind of database should I use? by Corion
in thread What kind of database should I use? by bart

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.