in reply to Garbage Collection on Hash delete

A word of warning: when using deeply nested HoH type structures with hundreds of key values per hash, the memory de-allocation process can take an inordinate amount of time (40 minutes in one case I had recently).

rdfield

Replies are listed 'Best First'.
Re: Re: Garbage Collection on Hash delete
by strat (Canon) on Jan 22, 2003 at 11:46 UTC
    Did that happen with perl5.6 or perl5.6.1?
    Some months ago I had the same problems with perl5.6 and 5.61 and decided to downgrade to perl5.005_03 (was the same behaviour under Solaris, Linux and Win2k (=AS 522)) and got RAM usage decreased from 800 MB to 500 MB, and Runtime from about 1:20 to 0:30. The data structure was a Hash of Hashes of Arrays (a simplified LDAP structure).

    But I'd like to run some tests about the behaviour of perl5.8...

    Best regards,
    perl -e "s>>*F>e=>y)\*martinF)stronat)=>print,print v8.8.8.32.11.32"

      My results were when using AS623 (5.6.0) on a 2GHz P4 with 512MB RAM. The data structure was a HoHoHoHoHoHoA (a total of approx 2.4M elements over ~ 600000 arrays), but sharing the RAM with a Oracle database (about 150MB resident in memory). Watching the datastructure destruct I saw 140000+ page faults. I'm looking to move the software to a Linux box, Perl 5.8 and put the DB on another box, and see how long things take.

      rdfield

Re: Re: Garbage Collection on Hash delete
by Elian (Parson) on Jan 13, 2003 at 18:19 UTC
    This is due entirely to a glibc bug in versions of glibc before 2.3. It had horrible performance when dealing with lots of small memory deallocations, and it will kill perl's performance in some cases. The easy solution is to upgrade your glibc if you can, which doesn't require any action for perl as it's a dynamically linked library, or build perl with perl's malloc if you can't.
      TVM. I experienced the problem with W2K, but I'll try it with a handy Linux box and get back to you.

      rdfield