Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^2: XML::Twig loves to eat my memory

by AndyZaft (Hermit)
on Jul 22, 2010 at 20:30 UTC ( [id://850906] : note . print w/replies, xml ) Need Help??


in reply to Re: XML::Twig loves to eat my memory
in thread XML::Twig loves to eat my memory

Something else just occurred to me is that in this case it might be beneficial to use twig_roots since you are only interested in 1 field anyway. Probably not going to help with the memory issues much in the end, but it might crash later :)

Replies are listed 'Best First'.
Re^3: XML::Twig loves to eat my memory
by carcus88 (Acolyte) on Jul 22, 2010 at 21:24 UTC
    twig_roots does prove to be most useful for when you only want one little bit of data. A single file only consumes 19MB as opposed to 166MB. However it still does not release memory when I try to $t->dispose(). This also does not solve my problem for scripts that have to process all the data.

      However it still does not release memory when I try to $t->dispose().

      How did you ascertain that?

        By watching the size of the process. It does not decrease after dispose is called. Unless I misunderstand the purpose of dispose I would expect memory to be released and the size of the process to go down.
      Very strange, since dispose calls the Twig object's destructor explicitly. But then again I don't know much about how the GC works in the Perl process. So your memory usage keeps growing the more files you process if I'm understanding this correctly, right?
        Correct, no matter what I have tried it keeps growing approx 160-180MB for each file.