LetoX has asked for the wisdom of the Perl Monks concerning the following question:

i've written a script that generates random german poems from a massive hash/ref-structure imported via Storable. To save the startup-time and lots of mem i converted the script to a mod_perl handler. it compiles and runs fine and initially it shares the complete mem (440 mb) of the apache process but after some runs the size of shared mem rapidly decreases. since i only read from the hash i think the mem gets dirty because of autovivification in the hash. i went over the code and added some exists() but it doesnt seem to change anything. after running for a while there is almost no shared mem left and the mem consumption gets crazy. my questions:
1. it would be very nice for me to declare the whole structure (multi-level/hashs/arrays/refs) read-only but readonly-pragma only supports scalars and i have no idea how to get constants to work with Storable. Anyone got an idea?
2. is there an autoviv of arrays. eg if i have an array with 4 elements and read-access the fifth, is the array changed?
3. is there a write in "foreach $elem (@{$ref})" to $ref?
4. if you are very nice and have absolutely nothing to do, would you throw a look at my source-code (www.versquelle.de, the page is in german but there is a quite understandable link to the source-code in the center of the page) and leave me a mail at info@versquelle.de.
help is very much appreciated.

Replies are listed 'Best First'.
Re: mod_perl shared mem and autoviv
by BrowserUk (Patriarch) on Jul 19, 2002 at 00:34 UTC

    Take a look at this and the thread it is a part of for the best explaination (thanks Merlyn) of this problem that I found.

    Note: If you typed "autovivication" into the search box this is what you would have found!

[reply]
Re: mod_perl shared mem and autoviv
by perrin (Chancellor) on Jul 19, 2002 at 00:43 UTC
    Here's an alternative: instead of importing the whole thing into memory, use MLDBM. It will require only minimal changes in your code, and should save buckets of RAM.

    It's very hard to avoid modifying Perl data structures (and thus unsharing them). Just reading a numeric scalar in a string context causes a change in the memory structure for that scalar. You may not be able to fix it.

[reply]
      in my first version, i used mysql and even with optimized indices the generation took several minutes. thats the cause i put the structure into mem, now it gets the thing done in 5 seconds.
[reply]
        MLDBM is not MySQL. Try it.
[reply]

Back to Seekers of Perl Wisdom