in reply to Replacing SQL with perl

In my experience, a lot depends on the size of your dataset. For a few hundred rows, unless you have very complex queries, a flat file is probably best. With complex queries it is much more a case of it depends. Depending on the table structure and the nature of the queries some complex things will be simpler in SQL, others in Perl. As a crude rule of thumb, if you must use a regex to search, use Perl.

For some applications I'm looking at, with 10,000 to 20,000, the time involved in reading the flat file and parsing it is significant. The well structured, binary format of db files makes reading them quicker, in a sense they are preparsed.

And finally, to repeat something someone mentioned earlier, it will be vastly easier to handle multiple scripts making changes at one time with a db then without.