XML::Simple is there a maximum xml file size?

webchalkboard has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: XML::Simple is there a maximum xml file size? by monkey_boy (Priest) on Jan 12, 2006 at 17:05 UTC
The maximum file size depends on your RAM size, XML::Simple creates a complex data structure, which in perl terms, will be much larger than the xml files size. The other thing to note is that XML::Simple can be very slow on complex/large xml documents, so it may be that your browser is timing out, before the parsing is finished, Id try running the script on the command-line to see if you get an "out of memory error" (from the OS), or to time how long the parsing takes. This is not a Signature...	[reply] [d/l]
Re^2: XML::Simple is there a maximum xml file size? by pileofrogs (Priest) on Jan 12, 2006 at 18:45 UTC
I'd like to second what monkey boy says in his second paragraph. Write a three line script that does nothing but open that file with XML::Simple, then you know it's not something else causing trouble. Make it work in the simplest possible environment, then put the solution into your working CGI.	[reply]
Re: XML::Simple is there a maximum xml file size? by aufflick (Deacon) on Jan 12, 2006 at 23:33 UTC
I've always found XML::Twig to be excellent for parsing large XML files - it also works on streams (ie. infinte sized XML files :)	[reply]
Re: XML::Simple is there a maximum xml file size? by johnnywang (Priest) on Jan 12, 2006 at 19:24 UTC
Just want to second what's been said, my experience with XML::Simple is that it's perfect for parsing such things as xml configuration files which are usually very small and not complex, try other parsers or regex if yours is either big or complex. In the two cases where I started with XML::Simple, both ended up with simple regex.	[reply]
Re: XML::Simple is there a maximum xml file size? by holli (Abbot) on Jan 12, 2006 at 17:41 UTC
over the whole 5.1mb XML Lucky you ;-) At work I have to process XML files that are five Gigabytes (!) and bigger. I can only praise the lord that they are program generated and have tight schemes, so I can do lots of stuff (safely) using regexes and state machines. holli, /regexed monk/	[reply]
Re^2: XML::Simple is there a maximum xml file size? by jalewis2 (Monk) on Jan 13, 2006 at 02:48 UTC
You must be working for the government. The only place I've seen that thinks 5GB of data is best formatted in a single XML file.	[reply]
Re^3: XML::Simple is there a maximum xml file size? by holli (Abbot) on Jan 13, 2006 at 06:34 UTC
You are close. Not for the government, but for a semi public authority. holli, /regexed monk/	[reply] [d/l]
Re: XML::Simple is there a maximum xml file size? by etm117 (Pilgrim) on Jan 13, 2006 at 15:55 UTC
The parser XML::Simple underneath the hood uses makes a difference also. If you leave it at the default, it will use XML::SAX. XML::SAX by default will use XML::SAX::PurePerl. As implied by the name, it is an XML parser written in Perl. For faster results, try installing XML::LibXML. You will need to install the library that it uses underneath the hood (libxml2) and it can be found here.	[reply]
Re: XML::Simple is there a maximum xml file size? by webchalkboard (Scribe) on Jan 13, 2006 at 12:56 UTC
Hi, Thanks for the responses, i've run the script on the command line and get an error now. `End tag mismatch (venue != primaryAct) [Ln: 2438, Col: 64]` [download] There must be something wrong with the structure of the XML file, i'm not sure. I will try and use the other module suggested and see if that works any better. Learning without thought is labor lost; thought without learning is perilous. - Confucius British Artists	[reply] [d/l]