aob_on has asked for the wisdom of the Perl Monks concerning the following question:

Hey all,

I have recently moved from 5.6.0 to 5.8.3 where I work and I have noticed something recently that I would like confirmation on and perhaps even a solution for.

It seems that large hashes under 5.8.x take up more memory than they did under 5.6.x, usually about 20% more memory. I have googled (groups and not), I have searched perlmonks, but nowhere can I find any reference to this issue. I did my own analysis looking for a reason and so far what I've come up with is that 5.8 seems to allocate memory using a more greedy algorithm but perhaps one that performs faster?

As an example, a program that creates a 500,000 key hash, we'll call it largeHash.

Under 5.6.0, largeHash when run makes 151 calls to brk(), increasing the memory allocated to the perl process. The increments in the address are quite small, seeming to indicate 5.6.0 requests less memory more often keeping its mem footprint low but perhaps at a bit of CPU cost... Conversely 5.8.3 running the code (again, largeHash) makes only **34** calls to brk() and the delta between addresses is much bigger, seeming to indicate less cost to the CPU at the cost of a larger mem footprint (and perhaps as a result a large segment of unused mem devoted to the process). Can anyone confirm or deny my conclusions? Am I just stupid?

Thanks!
aob

Update!!!! This appears to be JUST the perl process taking up the space (which I suspected but have now confirmed). Assuming Devel::Size can be trusted, under both perls the hash is taking the same amount of memory. I should mention too that in both cases perl was compiled with mymalloc set to yes. In other words using the system's malloc (HPUX 11.00).

Update 2: Solved, I relied on someone else when they told me the mymalloc settings between the 2 versions were the same. They weren't. I'm pretty sure it was just an issue of mymalloc and things are recompiling now, so I'll soon see. Thanks for the help!

janitored by ybiC: Retitle from less-than-descriptive "Looking for confirmation..." for better searching

  • Comment on *solved* Hash Memory Usage Changes in 5.8.x?

Replies are listed 'Best First'.
Re: Hash Memory Usage Changes in 5.8.x?
by perrin (Chancellor) on Jun 03, 2004 at 20:26 UTC
    If you want to save memory, make sure you have compiled with debugging and threads off. Red Hat's binary distribution of perl has these on, so I always recompile it.
Re: *solved* Hash Memory Usage Changes in 5.8.x?
by CountZero (Bishop) on Jun 03, 2004 at 22:37 UTC
    I don't know if it has anything to do with memory consumption but Perl 5.8.1. introduced a randomization of the order of the hash-elements from one run to the next. So something did change in the hash-code!

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law