Devel::Size is a very useful module for working out where your memory is being consumed, but it has one major caveat. In the process of determining the size of complex structures, itself consumes prodigious amounts of memory.

Most of that is consumed tracking references to a) prevent recursing endlessly around circular data structures; b) to avoid counting the same substructures multiple times. As all D::S requires is a boolean answer as to whether a given address has been seen (and counted) before, using a hash is a convenient but costly way to do it.

I've switch that to using a (segmented) bitstring, with the result that the following one-liners require a tiny fraction of additional memory for D::S to do it's thing, relative to the current cpan version, as demonstrated by the following one-liners:

c:\test>perl -MDevel::Size=total_size -lwe
"$h{ $_ } = [1 .. $_] for 1 .. 1e3;scalar<>;print total_size( \%h ); s
+calar <>;"
11,196k
10115049
46,260k

c:\test>perl -MDevel::Size=total_size -lwe
"$h{ $_ } = [1 .. $_] for 1 .. 1e3;scalar<>;print total_size( \%h ); s
+calar <>;"
11,196k
10115049
11,240k

c:\test>perl -MDevel::Size=total_size -lwe
"$h{ $_ } = [1 .. $_] for 1 .. 5e3;scalar<>;print total_size( \%h ); s
+calar <>;"
253,664k
250591721
1,147,000

c:\test>perl -MDevel::Size=total_size -lwe
"$h{ $_ } = [1 .. $_] for 1 .. 5e3;scalar<>;print total_size( \%h ); s
+calar <>;"
253,644k
250591721
253,776k
[download]

The 3 numbers below each run are: 1) The memory consumed by the process prior to sizing it; 2) the size of that structure as evaluated by D::S; 3) the memory consumed by the process after D::S::total_size() runs. The different between the first and third figures is the overhead of the tracking mechanism.

The first set of figures in each pair are those for the cpan version 0.71; the second for my modified version (dubbed 0.71x for now)

As you can, in these tests, rather than incurring an overhead of ~4 times the requirement of the data-structure being measured, it is just a couple of hundred KB in both cases.

This post is to ask for testers on other (non-win32; 32-bit only currently*) platforms that think they have existing testcases that will exercise this well, to email me at vev6s4702 sneakemail.com (complete) and I'll send them a new distribution (30k) for testing.

(*)I hope to extend that to 64-bit too, but will need the cooperation of someone with a (preferably win64) suitable platform.