Re^2: Libxml parser cosuming 100% cpu

Thanks, (first point) But i think if i use XML::Twig then execution time will be more than 2.5min. second thing i got it XML::LibXML store the doc in memory but its a small record of 100 Lines around ? do you think XML::Twig reduce the cpu usage and fast the execution. (third point) Which findnodes you are talking about.

i have some more question here Which one is fast there in each two opt
+ion:
1).
$method = $xml->getChildrenByTagName('Method')->to_literal;
$method = $xml->fidnnodes('./Method')->to_literal;

2).
$xml->findnodes('./Indi/Lost/true')
$xml->exists('./Indi/Lost/true')
[download]

Comment on Re^2: Libxml parser cosuming 100% cpu Download Code

Replies are listed 'Best First'.
Re^3: Libxml parser cosuming 100% cpu by haukex (Archbishop) on Aug 11, 2018 at 13:37 UTC
But i think if i use XML::Twig then execution time will be more than 2.5min. When it comes to optimizations, I'd suggest not "guessing" which might be faster, but measuring and testing! i got it XML::LibXML store the doc in memory but its a small record of 100 Lines around ? do you think XML::Twig reduce the cpu usage and fast the execution. If I understand correctly that you're splitting your ~1 million line file into chunks of 100 lines and then processing those one at a time, then I would agree with the guess that XML::Twig might not give you a big speed boost (unless you have ridiculously long lines). On the other hand, if your ~1 million line file is one big, well-formed XML file, then you would be able to get rid of your custom "splitting" code and use XML::Twig to process the entire file, one "record" at a time. If you need help with that, you'd have to show us some sample input (see How do I post a question effectively? and Short, Self-Contained, Correct Example). Which findnodes you are talking about. I was talking about this: `if ($xml->findnodes('./Indi/Lost/true') ) { } if ( $xml->findnodes('./Indi/Lost/true') \|\| $xml->findnodes('./Indi/Lo +sgshshsht/false') ) { }` [download] Which is better written like this, to avoid the doubling of the `findnodes` call (Update: unless of course the first `if` block makes modifications to the document that would require the second `if` to re-run the `findnodes`): `my $result = $xml->findnodes('./Indi/Lost/true'); if ( $result ) { } if ( $result \|\| $xml->findnodes('./Indi/Losgshshsht/false') ) { }` [download] And a similar thing with `$xml->findnodes('./nike'))[0]->firstChild`. Which one is fast there in each two option I don't have the time to test right now, but the go-to module for this kind of comparison is Benchmark. But as I said before, measure where your code is spending the most time with Devel::NYTProf, and then optimize those places, instead of guessing and doing what might turn out to be an unnecessary micro-optimization.	[reply] [d/l] [select]
Re^3: Libxml parser cosuming 100% cpu by ikegami (Patriarch) on Aug 11, 2018 at 12:49 UTC
If your saying your XML doc has only ~100 nodes, then the problem isn't memory, then using a pull parser like XML::Twig (or XML::LibXML::Reader) won't help.	[reply]