Re^4: extracting data from HTML

forgive me my brethern but it looks I did bite off more then I could chew and again I endup with bits I can not put together

the example of
my $nodes = $xml->findnodes('//*[local-name()="title"]')
wasn't to much to understand, altohug quite surprised with the construction of the xpath-expression; I would had expected something more easy like
my $nodes = $xml->findnodes('//html/head/title]')
But ofcourse, it wouldn't be me if I would get it wrong again

With HTML::TreeBuilder::XPath it did work, even things like giving me all the table rows from a specific path and dump as text with

my @stuff =
    $tree->findvalues( '//td[@class="BTrow"]/table/tr/td/table/tr');
print Dumper(\@stuff);
[download]

Trying that with HTML::HTML5::Parser only resulted in undefined results

It looks to me I'm missing some bit

please toby, and others as well, what am I doing wrong here, it can't be the xpath syntax, is it?

thank you all for your enlighting words and inspiration

Comment on Re^4: extracting data from HTML Select or Download Code

Replies are listed 'Best First'.
Re^5: extracting data from HTML by tobyink (Canon) on Jun 05, 2012 at 21:59 UTC
The problem with this: `$xml->findnodes('//html/head/title')` [download] ... is that none of the names in the path include a namespace. (X)HTML elements are always namespaced. Hence my rather awkward... `$xml->findnodes('//*[local-name()="title"]')` [download] i.e. "select all elements where the local part of the element's name is 'title'" Another solution (arguably a lot more readable) would be to forcibly bind the XHTML namespace to a prefix: `$xml->documentElement->setNamespace( 'http://www.w3.org/1999/xhtml' => 'xh', );` [download] And then you can freely use that prefix in XPaths. `$xml->findnodes('//xh:html/xh:head/xh:title')` [download] This specific problem is mentioned in the XML::LibXML::Node documentation - look for the "NOTE ON NAMESPACES AND XPATH" in the documentation for the `findnodes` method. `perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'`	[reply] [d/l] [select]
Re^6: extracting data from HTML by Jurassic Monk (Acolyte) on Jun 06, 2012 at 15:37 UTC
that is pretty awkward! strangely enough it seems that the other modules do not bother about the namespaces. The reasoning behind the namespaces and that those thus need to be included in the XPath is clear. Parsing the badly formed HTML into a XML-structure should ofcourse lead to proper XHTML, I do see that. Hence nsURI does use http://www.w3.org/1999/xhtml. I made a dump of the 'title' node: `'nsuri' => 'http://www.w3.org/1999/xhtml', 'suffix' => 'title', 'qname' => 'title', 'children' => [ 'extracting data from HTML' ], 'type' => 'Element', 'attributes' => [], 'prefix' => undef` [download] As the webpage of the referenced perlmonks node is just plain html with `<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">`, I will asume, somewhere the `http://www.w3.org/1999/xhtml` must have slipped in. It would be nice to override that - or, it would be nice to just remove it from each and every node in the XML-Document. Maybe there is a way to bypass those default namespaces, or do some quirks on the XPath expression, before applying it to the document All in all... yes I do desire to be fully knowlageable off all ins and outs that come with XML, XML-Schema, XPath, XSLT and more... But I don't apreciate this awkward namespace trouble Brethern monks, is there a way to get around this, other than inseting my own prefix? N.B. almost every example off XPath expressions, this is overlooked and simplyfied, which is a nasty bummer when you want to work with it	[reply] [d/l] [select]