hraj has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I need to read Huge XML file. I have tried using XML::Simple and XML::DOM, but both of them failed giving me some Application Error while parsing.

Will some one suggest me any module which will read Huge XML file.

Note: I don't want to use regular File IO operations to read XML file.

Thanks,
Hemaraj

Replies are listed 'Best First'.
Re: Processing Huge XML file
by davido (Cardinal) on Dec 14, 2005 at 06:50 UTC

    You want to be using XML::Twig: "A perl module for processing huge XML documents in tree mode." (from the POD)


    Dave

[reply]
Re: Processing Huge XML file
by murugu (Curate) on Dec 14, 2005 at 08:46 UTC
    Hi hraj,

    Best bet would be XML::Twig.

    Regards,
    Murugesan Kandasamy
    use perl for(;;);

[reply]
Re: Processing Huge XML file
by planetscape (Chancellor) on Dec 14, 2005 at 14:20 UTC

    XML::Twig, definitely. There are many examples and a comprehensive tutorial here. Don't forget about Super Search; typing "xml large" in the "Match text containing" field would turn up many useful recommendations.

    HTH,

    planetscape
[reply]
Re: Processing Huge XML file
by ghoti (Acolyte) on Dec 14, 2005 at 15:07 UTC
    I don't know how huge "Huge" is but I'm parsing a file that's about 200 MB and decided to go with the lower level XML::Parser and wrote my own handler. This allows me to have greater flexibility in handling the data.

    On a Linux box with 4 GB it takes about 20-25 minutes. Note that if you do store all those data in memory (e.g., using DOM or a roll-your-own solution), you better have enough memory to handle it all.
[reply]

Back to Seekers of Perl Wisdom