Re^5: Constructing a hash - why isn't my regex matching anything

Well, I'll be....

Any idea of where all that extra memory usage is coming from (beyond the 78M for Data::Dumper)? That's a lot of extra space for 13M of actual data. Based on a conversation in the CB, hash buckets only account for about half a meg extra, not 60M (or 20M as per another tester in a reply further up)

Update:A quick check on my machine comes up with 26M for storing key value pairs in an array, and 34M for storing them in a hash:

   key-value pair: 112 bytes
   total data for 125,000 key-value pairs: 13.25M
   virtual memory usage for array built via push @aData, $k, $v:  26M
   virtual memory usage for hash built via  $hData{$k} = $v:  34M
[download]

The test script is below

use strict;
use warnings;

my $k="//programfiles/documents/data/lookup/script_auth_pap.h";
my $v="\\root\\edit\\perl\\scripts\\scripths\\sec\\inc\\script_auth_pa
+p.h";

# -----------------------------------
# pick one or the other array or hash
# -----------------------------------

#my @aData;
#for (1..125000) {
#  push @aData, $k, $v;
#}

my %hData;
for (1..125000) {
  $hData{$k}=$v;
}

# -----------------------------------
# memory consumption output
# -----------------------------------

my $iPair = length($k) + length($v);
printf STDERR "Length of key-value pair: %d bytes; total data=%.2f M\n
+"
  , $iPair, (125001*$iPair)/(1028*1028);

warn `ps -o vsz -p $$ 2>NUL`, ' ';
[download]

Comment on Re^5: Constructing a hash - why isn't my regex matching anything Select or Download Code

Replies are listed 'Best First'.
Re^6: Constructing a hash - why isn't my regex matching anything by Anonyrnous Monk (Hermit) on Dec 19, 2010 at 12:59 UTC
See illguts and add up the individual HEs, HEKs, SVs, etc. A simple scalar value (SV) already has 4 fields plus the actual data body. And even for the array data type, which is less complex than a hash, an empty entry adds up to around 80 bytes administrative overhead.	[reply]
Re^6: Constructing a hash - why isn't my regex matching anything by Anonymous Monk on Dec 19, 2010 at 12:57 UTC
I have no idea :) Maybe its because i have a 64-bit cpu, running 32-bit OS with 32-bit perl, with only 512mb physical memory .... I think its all the virtual memory managers fault :) at least its better than 5.6.1 :) Read more... (4 kB) mingw32/msvs6 doesn't make a difference	[reply] [d/l]