in reply to Re: Re: Question about properly laying out a database
in thread Question about properly laying out a database

But if he's going to search all the records to match on some set of criteria, won't he have to load the whole thing in memory anyway?

In this application, it's not very often that you call up one record by its key id. Searches are more common. So, you'll have to evaluate just how much data each record will hold, how much RAM will be used up, and how quickly you can load all that data into memory.
  • Comment on Re: Re: Re: Question about properly laying out a database

Replies are listed 'Best First'.
Re: Re: Re: Re: Question about properly laying out a database
by perrin (Chancellor) on Dec 12, 2001 at 12:32 UTC
    But if he's going to search all the records to match on some set of criteria, won't he have to load the whole thing in memory anyway?

    No. All he has to do to find the cars with make = 'volvo' (or some normalized key like 'make_7') is say my $cars = $make_db{'volvo'} and have that return a ref to an array of car IDs (serialized by Storable). Then he does the same for each of the other criteria, and finds the overlapping set.

    You do access records by ID, because you make multiple indexes (dbm files) which are each using a different criterion of the search as a key. It's kind of a roll-your-own MySQL.

      You do access records by ID, because you make multiple indexes (dbm files) which are each using a different criterion of the search as a key. It's kind of a roll-your-own MySQL.

      Isn't it also overkill?

      Given the claimed upper bound of 500 records (which means be generous and assume an upper bound of 1,000), it's still going to take fewer reads (and fewer disk head movements) to suck in an entire flat file of search criteria than it would to search using multiple DBM files.

      There's a point at which the multiple DBM approach is a win, but I suspect this application is far below that point.

        Kind of depends on the size of the car records, but it could be. I missed the part about only having 500 records. It would be interesting to try DBD::Sprite and see how it does on this.