in reply to Re^5: Does "preallocating hash improve performance"? Or "using a hash slice"?
in thread Does "preallocating hash improve performance"? Or "using a hash slice"?

I can't see any value going further.

I actually wasn't expecting you to get back on that explanation with so much detail and technical information, so thanks for that.

The part I got wrong is that I thought you meant that all the data (all the strings, not just the SV*s) was duplicated four times; this is also why I started talking about COWs, because I didn't understand why perl would need to copy the strings so many times. I got confused by "two copies of all the keys and values" where I failed to understand that "keys and values" was refering to their SV*s. So by "4 times more memory", I meant 4 times more than the total_size of the hash, not just the structure size.

So, for those occasions when the destination list is entirely defined by one array, and the source list entirely defined by another array, it would be possible to only pass references to those two arrays on the stack, and thus avoid large scale allocations; but is would require special casing, and probably another opcode.

This was what my "even more so with arrays" was about, I didn't understand the need to have all the data duplicated so often. Again, it's the "all the data" I got wrong. Indeed my post doesn't make much sense starting from that point.

Your expectations are wrong.

Clearly I've been underestimating the proportion of structure data over actual data. Though you did say yourself that keys strings are always copied into the hash. But they are not allocated or copied 4 times. With $keys[ $_ ] = "$_" x 20 in your code (so strings with a mean length around 100 characters) I get:

8 096 Ko 3 228 772 Ko 6 396 092 Ko 2091995858
Where 6396092*1024/2091995858 = 3.131.

Thanks again for taking the time to detail your answer.

Nb: I got the mem sub in BrowserUK's post here if anyone is interested. The numbers are separated by \xFF (not a space) so /(\S+ Ko)$/ worked for me.