If you're using XML::Simple and it's slow, the first thing to check is what parser you're using. If you have version 1.08_01 of XML::Simple and you have XML::SAX installed then you may using XML::SAX::PurePerl which is slow. Look for ParserDetails.ini in the site_lib/XML/SAX directory. All your installed SAX parsers are listed in this file and the last one listed will be used by default. If you don't already have XML::LibXML installed then that's the fastest XML parser available and XML::Simple will use it if you have SAX installed.

I'd also suggest a couple of tweaks to your code to squeeze a little more efficiency out of it. Both the <field> and <child> elements are being folded into hashes (on the 'name' and 'id' attributes respectively). You are then processing every element in each of these hashes by iterating over keys. A more efficient way would be to set the option keyattr => {} on the call to XMLin() to disable folding. Then you can iterate through the resulting arrays (rather than hashes) with:

foreach ( @{$node->{data}{field}} )
{
    $template->param( $_->{name} => $_->{content} );
}
[download]

For maximum efficiency, you'll want to ditch XML::Simple altogether and refactor your code into a SAX handler.

From what I understand from the snippet, your script takes a "node" (xml file), renders html based on its content and recursively calls itself for every "child" element encountered (the child element references another xml file).

At the end, your sub returns a flattened HTML string (if I interpret HTML::Template's output() method correctly). Both the XML::Simple object and HTML::Template object go out of scope.

That means, that during the recursive process the max. amount of memory depende on the total length of the outputted HTML string, together with (the recursion depth) x ((the memory needed for a XML::Simple instance) + (a HTML::Template instance)). From your example XML snippet, this looks not like a big deal. I'm not familiar with XML::Simple's internals, but being based on a SAX parser (expat) I guess that after parsing mostly the Hash of Hashes/Lists of the $node object remains.

Whether this implementation is too slow depends largely on the nesting depth and your definition of "slow", but in order to save on both memory and CPU, you could conceivably render these discussion threads to static files in a background process, on regular intervals or as soon as a new item has been added to the discussion. Or even on first request, from a nifty 404 error handler ;-)

Just my $0.02

--
Cheers, Joe

This doesn't answer your question directly, but I think it is relevant nevertheless.

Actually, this is a job for XSLT, the XML transformation language. The idea is that one describes a transformation of XML into "something else" which can be either XML, HTML, plain text, basically whatever you like.

If the XSLT modules on CPAN are worth their salt, they should be quite efficient, so maybe this is an occasion not to reinvent the wheel ;-)

XSLT is a W3C standard since a couple of years now, so the technology is quite mature.

Hope this helps, -gjb-

My XSLT experience is fairly light so I'm keen to find out about real world applications. In the original poster's code he would encounter an element like this:

<child id="2">435</child>
[download]

At this point his code would then (recursively) parse down into the file called '435.xml'. I realise recursive processing is no problem for XSLT, but how would you load in another file and process it?

Oops, I should have read the code more carefully. I thought it was just an XML file that was converted to HTML without recursion.

Recursion complicates matters to the point that you have to do major hacking in order to get this to work with XSLT if it is possible at all.

Thanks for pointing this out, -gjb-