XPath searching too slow

dietbuddha has asked for the wisdom of the Perl Monks concerning the following question:

Is there some modules we can swap out HTML::TreeBuilder and XML::XPathEngine which are either faster or will at least push more of the cost into the parsing and creation of the tree instead of the searching?

We've recently started testing our dynamically generated HTML. However it's taking much longer now for the test suite to complete. We did some profiling on our slowest test file and found our 95% percent of the time spent in XML::XPathEngine which is the searching we do. However, less than half the tests are HTML related.

Comment on XPath searching too slow

Replies are listed 'Best First'.
Re: XPath searching too slow by mirod (Canon) on Jul 31, 2008 at 17:52 UTC
First, are your queries optimized? It is very easy with XPath to write a very expensive query that needs to traverse the entire tree (think `//p`). Then are your tests separate processes? XML::XPathEngine caches the parse tree for queries, but only within the same process of course. That might be one reason why the tests run slower. That said it the XPath engine _is_ rather slow. Patches or replacement engines are welcome ;--) One way I can think of, to speed it up, might be to allow placeholders instead of values in the queries.	[reply]
Re: XPath searching too slow by moritz (Cardinal) on Jul 31, 2008 at 17:17 UTC
XML::LibXML can search for XPaths, and since it's mostly a wrapper (a very good one, that is) around a C library it could be quite fast. But of course you'd have to benchmark that first. If you need to make many queries, you can also build relational indexes for xpath, but that's likely to require quite a lot of work. Or maybe you could just use a different templating technique.	[reply]