in reply to Anti-slurp weblog

There is also great merit in maintaining an index file. You can index, say, the date, title and filename, and the index file will probably remain small enough that you *can* slurp it. I mean, you're not going to run Slashdot with it but even if you post once a day for the next three years you're still looking at a few K. And you can do a few neat tricks like just reading off the top three records to save time.

For great speeedness, use fixed-length records. Space wastage sucks, but you then get the luxury of being able to arbitrarily jump around the file.

<lecture props='grep'>
The thing is, once you you've got a good index file system going, you probably want to add querying abilities, so you'll add in a keyword index. Then you'll break the index handler off into a daemon process so you can update asynchronously and handle collisions. Then you'll generalise the data storage to handle arbitrary data in arbitrary table. You might even add a SQL interpreter. You get the idea - you will thank yourself for learning to use a database and DBI.
</lecture>

But if it's impossible, I meant what I said about an index file.

Replies are listed 'Best First'.
Re: Re: Anti-slurp weblog
by newrisedesigns (Curate) on Jan 22, 2002 at 06:12 UTC

    Everyone++ for all your help. The Index file idea is great.

    The problem is, I'm virtually hosted, and I don't have mySql or postgres set up. Although I can have my website 'upgraded' (my service provider is updating servers), I don't have the time to back everything up and move everything over. Mark it up to plain old laziness and lack of time.

    Hopefully though, I will be on the new server, and I will rewrite the weblog using mySql (offered with the upgrade).

    Thanks again for all your help, it is greatly appreciated.

    John J Reiser
    newrisedesigns.com