in reply to To learn to search flat files or to cheat...

If you're working with a flat-file, using it to represent records, you've got some issues to deal with. First, if the individual records aren't fixed-length, then you need to maintain an index, or suffer the performance penalty of simply having to skim the entire file just to find one record. Additionally, any change you make (again, assuming the records are not of fixed length) will mean you have to rewrite the file, or mark an existing record null and void, and append its replacement at the end of the flat file (which further hampers searchability). In other words, a non-fixed-length flat file based records approach is kinda messy and speed inefficient.

On the other hand, flat files with fixed length records would allow for rewriting individual records without rewriting the entire file. Still, an index file would be a good idea, unless you maintain the flat file in some sort of sorted order so that you could perform binary searches on it when you need to find something.

All this is usually too much work. DBD::SQLite is really a convenient alternative, and nearly as compact as a flat-file approach, but without all the complexity. Honestly, you don't need to do this yourself, unless you're sitting in a comp sci class.


Dave

  • Comment on Re: To learn to search flat files or to cheat...

Replies are listed 'Best First'.
Re^2: To learn to search flat files or to cheat...
by stonecolddevin (Parson) on Nov 10, 2006 at 21:39 UTC

    I agree, SQLite is everything I need right now, even moreso that MySQL in that it's much more convenient, compact, and takes next to no memory. I'm not up to commercial apps yes, so I don't need something that's going to eat at my memory like MySQL does sometimes.

    I am having trouble with read only errors after I uploaded my database to my server, should I just use SSH and create a new database with the same columns and such? That way I don't think the readonly errors will pop up when I go to write to it...

    meh.
[reply]

      You could just create a new one there. What do you suppose is the problem? Did you transfer the file in ASCII mode when it should have been in binary mode, or vice versa?

      I've never tinkered with the portability across OS's of the file that SQLite uses to store its data. But I think SQLite does support block inserts, so you could write a script that just exports all the data into an ASCII file, and then another one to import it into SQLite at the other end of the line.


      Dave

[reply]
        I've never tinkered with the portability across OS's of the file that SQLite uses to store its data.
        I've always heard that SQLite files are cross-platform compatible, it's only cross-SQLite-version compatibility that is an issue. dhoss should try to figure out the SQLite versions the DBD's are built against, or that his SQLite browsing program is built against. If they don't match, that will quite likely be the problem. (Unless, like you said, he botched the upload. Which I doubt.)

        p.s. I looked if I could find some official docs on file compatibility, and I found this on The SQLite introduction page:

        Database files can be freely shared between machines with different byte orders.
[reply]

        I'm not sure actually. I think I'll probably just create a new DB via SSH. I could tinker around with it and see if I could get the existing one to work, but I'd rather set it up so that there's a setup method run if no DB file exists, much like you said.

        So where do I find out what format the SQLite db needs to be in in order for me to export my data into a new database?

        meh.
[reply]

In Section Seekers of Perl Wisdom