in reply to complex string matching

And if you are looking for “a disk file,” don’t overlook SQLite. http://www.sqlite.org

It is an extremely fast, public domain, flat-file database system.   (The only “gotcha” might be that you probably want to do things within a transaction, because when you don’t, SQLite carefully ensures that everything has been written to disk.   Exactly as it should, of course, but it slows down bulk-operations considerably.)

So...   where you might code your own logic, have a tied hash and so on, you might be able to use a query ... the result being that it is just as fast as it was for the computer, but considerably faster for you.

Replies are listed 'Best First'.
Re^2: complex string matching
by Your Mother (Archbishop) on Nov 04, 2010 at 13:49 UTC
    but it slows down bulk-operations considerably.

    As I understand it, this isn't quite right. It is slower for multiple transactions but single bulk transactions are faster than Pg and MySQL and since most operations are faster, SQLite comes off as the clear winner for speed: SQLite speed (see disclaimer on age of benchmarks and IANADBA).

[reply]

      Oh, no question at all about that.   And, no question that SQLite is doing exactly what it should.   But the difference can be quite dramatic (and of course, they stress to say as much.)

      SQLite is naturally very fast because it is writing directly to a file.   There is no IPC-protocol overhead no matter how slight.   But it takes a very cautious about ensuring that disk-writes really have happened.   And this slows the processing down to a speed that is determined by the rotation-time of the disk platter.   Whereas, if a transaction is in effect, it knows that it doesn’t have to do that.   Since client-server databases might handle write-commits in a different way (vis-a-vis what they do and don’t oblige the client to stop and wait for).   I meant it just as a little heads-up...

      SQLite is one of those software projects that makes you sit up and bow down.   A real, “holy smokes!!” piece of software.   (Y’know, like Perl... and I mean that.)

In Section Seekers of Perl Wisdom