Re: Memory errors while processing 2GB XML file with XML:Twig on Windows 2000

You don't give us much to work with. Some code and perhaps the amount of RAM on your system would help.

If you do a straight XML::Twig->new->parsefile( 'my_big_fat_xml_file.xml');, then the resulting data structure should need somewhere around 20GB. That's why XML::Twig let's you process a file one chunk at a time, and purge the memory when you're done with it.

The README for the module (at least for the latest version) includes links to lots of resources about the module. You could start by looking at xmltwig.com.

Comment on Re: Memory errors while processing 2GB XML file with XML:Twig on Windows 2000 Download Code

Replies are listed 'Best First'.
Re^2: Memory errors while processing 2GB XML file with XML:Twig on Windows 2000 by nan (Novice) on May 16, 2005 at 11:03 UTC
Hi, If you don't mind, please refer to the codes and XML segment in my replies to other people. My RAM is 1GB and my perl version is the latest. Initially, I have a virtual memory error too but it's solved after I changed virtual memory to the maximum (4GB), now I only have experienced a memory writen error. Do you think it could be better if I try linux? Thanks again,	[reply]