in reply to arrays in database

If I correctly understand your question, you would want to create a table in the database for each of your existing text files. To suck the entire contents out of a table, you would run a query against it (most likely SELECT * FROM tablename;, if you really do want everything) and use the query object's fetchrow_array method in list context.

However, if you do that, the performance improvement isn't likely to be much on reads or appends, although deleting data from the start of the list would probably improve considerably. Sucking an entire file into memory tends to be faster than sucking an entire database table into memory for the simple reason that it's going through a less complex access mechanism. The major advantages of databases are in seeking data (so you don't have to bring everything into memory) and concurrency, and slurping the whole table doesn't take advantage of either of them.

So, what do you actually do with the data after reading it? Do you really need to have everything in memory or can you use SQL to specify a subset of the data which is actually interesting to you?

Replies are listed 'Best First'.
Re: Re: arrays in database
by dev2000 (Sexton) on May 18, 2002 at 16:55 UTC
    I think the table itself would be a listing of hundreds or thousands of users...
    speeding up multi-player game with shared memory or MySQL
    ...with one column for the name (primary key), another for time and status data, and finally the array, each line of which represents game data communication between players.

    That's the way it is now as many flat files. :)
      So you're asking about how to store something along the line of perlmonks' chatterbox messages? In that case, I wouldn't even bother with writing something that transient to disk. The amount of content would likely be small enough to use shared memory for it, or, if that gets to be too much of a problem, you could use an in-memory filesystem (such as linux's tmpfs; I don't know if freebsd has an equivalent) with either flat files or a temporary database that gets created on server startup. If you go with flat files, remember mbox and maildir: You'll get better performance (especially on deletes) by chunking the messages into multiple files (say 1/message or 1/minute of messages) instead of using one big file for everything.
        Someone suggested using MySQL tables of type HEAP (I was thinking shared memory), I think this is a GREAT idea for all of the temporary information like user status and time, but the actual game data needs to be able to be stored in a "command stack" array so when the user does (sooner or later) pick it up it, the commands can be executed in the exact order they were received (to properly update the game map, etc). Also, the multi-player game world persists even after the player stops playing. So next session he will need to update his map accordingly (if he was affected while he was offline). But I'm still not clear on how to best accomplish this last part in any other way besides using one flat file (appending commands to the command stack) per user.

        I can't wait to finish and invite everybody!