jmo has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, I parsed XML into an array/hash ref 'struct' and tried to acces the data wrong, ie treating a hash ref as an array ref or vice versa. The result was that the perl process ate all memory which, of course, isn't a very nice thing.

Is there a good way to avoid thoose pitfalls, except using more simple data structures and not coding wrong?
Here's an example that consumes all my memory at least:
perl -e 'use strict; my $a = { child1 => [ {child2 => [ { child3 => [ 
+{child4 => [ 1 ] } ] } ] } ] }; print $a->{child1}{child2}{child3}{ch
+ild4}, "\n"'

[download]

Replies are listed 'Best First'.
Re: Perl eats all memory when accessing hash/array refs wrong
by Abigail-II (Bishop) on Sep 25, 2003 at 10:56 UTC
    The problem is that you are using a reference as index in an array. A reference used in numeric context returns the memory address of whatever this is pointing to. This usually is a big number, which means that Perl is going to create an array large enough to index on that position. Which will result in allocating lots of memory.

    The effects are system dependent, and if you have lots of memory, and the reference happens to be a relative low address, you might even be lucky enough that Perl will be able to allocate the memory.

    Abigail

[reply]

      The problem is that you are using a reference as index in an array.

      I admit that this appears to be whats going on, But I do not understand how. I see no lookup of an array in that code.

      Heres some toying I did with the problem. 

      #!perl -l
use strict;
use warnings;

my $a = {child1 => [ {child2 => [ {child3 => [ {child4 => [1]} ]} ]} ]
+};
print $a->{child1}{child2}{child3}; #cut off {child3} and no problem.

print '$a:',ref $a;
print '$a->{child1}:',ref $a->{child1};
print '$a->{child1}{child2}:',ref $a->{child1}{child2};
print '@{$a->{child1}}:',scalar @{$a->{child1}};
__END__
$a:HASH
$a->{child1}:ARRAY
$a->{child1}{child2}:HASH
@{$a->{child1}}:28004977

      [download]

      So it would appear that somehow $a->{child1}{child2} is being treated as an array lookup. But how? There is no array lookup involved.

      Id be interested to hear an explanation as to why this isnt a bug in perl.


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi
[reply]
[d/l]
[select]
        Pseudo-hashes are an experimental and soon-to-be-deprecated feature.
[reply]

      The problem is that you are using a reference as index in an array.

      How so? You can certainly see the effect with the attempt to access the first non existent key (but only on Linux) but could you explain that further..... 

      C:\>type test.pl
use Data::Dumper;

my $a={ child1 => [ {child2 => [ { child3 => [ {child4 => [ 1 ] } ] } 
+] } ] }; 
print Dumper $a;
print Dumper $a->{child1};
print Dumper $a->{child1}{child2};
print Dumper $a->{child1}{child2}{child3};
print Dumper $a->{child1}{child2}{child3}{child4};

C:\>perl test.pl
$VAR1 = {
          'child1' => [
                        {
                          'child2' => [
                                        {
                                          'child3' => [
                                                        {
                                                          'child4' => 
+[
                                                                      
+  1
                                                                      
+]
                                                        }
                                                      ]
                                        }
                                      ]
                        }
                      ]
        };
$VAR1 = [
          {
            'child2' => [
                          {
                            'child3' => [
                                          {
                                            'child4' => [
                                                          1
                                                        ]
                                          }
                                        ]
                          }
                        ]
          }
        ];
$VAR1 = undef;
$VAR1 = undef;
$VAR1 = undef;

C:\>

[root@devel3 root]# cat test.pl
use Data::Dumper;

my $a={ child1 => [ {child2 => [ { child3 => [ {child4 => [ 1 ] } ] } 
+] } ] }; 
print Dumper $a;
print Dumper $a->{child1};
print Dumper $a->{child1}{child2};
print Dumper $a->{child1}{child2}{child3};
print Dumper $a->{child1}{child2}{child3}{child4};
[root@devel3 root]# perl test.pl
$VAR1 = {
          'child1' => [
                        {
                          'child2' => [
                                        {
                                          'child3' => [
                                                        {
                                                          'child4' => 
+[
                                                                      
+  1
                                                                      
+]
                                                        }
                                                      ]
                                        }
                                      ]
                        }
                      ]
        };
$VAR1 = [
          {
            'child2' => [
                          {
                            'child3' => [
                                          {
                                            'child4' => [
                                                          1
                                                        ]
                                          }
                                        ]
                          }
                        ]
          }
        ];
$VAR1 = undef;   # hangs for 5 seconds here
$VAR1 = undef;
$VAR1 = undef;
[root@devel3 root]#

      [download]

      cheers

      tachyon

      s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[reply]
[d/l]
Re: Perl eats all memory when accessing hash/array refs wrong
by tachyon (Chancellor) on Sep 25, 2003 at 10:51 UTC

    Don't do that then :-) However it generates no output (as expected), but also causes no issues for me on Win32 or RH 7.3 both with 5.6.1. The behaviour on Linux IS subtly different though. On Win32 it just execs it and immediately exits. On Linux it has a runtime of about 5-6 seconds and uses 25% of 1 CPU on a Quad 2G Xeon machine. No notable memory footprint watching with top though. It would seem to be getting itself in some sort of loop on Linux, but as for why or what or how I have NFI.

    Read more... (7 kB)
[reply]
[d/l]
Re: Perl eats all memory when accessing hash/array refs wrong
by graff (Chancellor) on Sep 26, 2003 at 05:01 UTC
    There are many ways to get perl to consume all available memory -- and some, like the method discussed here -- are quick and easy. Well, heck, these things happen, and "a good way to avoid them" is, as you have said, to keep thing simple and not code wrong.
[reply]

Back to Seekers of Perl Wisdom