in reply to Comparing memory requirements for hashes and arrays
Questions to think about for hash versus array:
Is your data rectangular?
Regular database tables
without NULLs nicely fits a rectangular data structures.
Irregular data fits better in hashes.
What sort of iteration will you need to do?
If you will know the key to find your data element,
a hash is much better than searching over a list.
If you need to do a comparison on each key anyway, an
array might be easier to search. For example, if you
are checking each key to see if it matches a regular
expression, an array can be better. (Let me know if you want
to know why :-).
Are the keys of the hash simple to implement?
If you
have one database field that is the key, it is easy to use
it as a hash key. If the database rows are keyed with
multiple columns, the hash gets more complicated since you
will need to combine columns to make the hash key.
Do you need to keep reloading your data structures from
the database, or are they static?
If they are static,
you can use a few tricks that save memory.
If you have a machine with shared libraries
and
copy-on-write virtual memory, you can get multiple
modperl http daemons to share the database data.
You can presize arrays so that they have less memory overhead. For measuring memory consumption, the normal tools such as ps and top will work reasonably well.
To see if high-level behavior such as copy-on-write is working properly, you need to stress test your server to see how much traffic load causes it to swap. You really need to use a development server for this type of testing.
Writing a stress-test program is fun! Create a program using LWP to simulate the behavior of a single user. Run a bunch of these programs at the same time to simulate the load caused by many users. You can get typical user behavior patterns by examining your server log files. This approach allows you make impressive claims, such as "This system is scaled for two second page load times with 1000 simultaneous users."
It should work perfectly the first time! - toma
|
|---|