Re^5: extracting data from HTML

The problem with this:

$xml->findnodes('//html/head/title')
[download]

... is that none of the names in the path include a namespace. (X)HTML elements are always namespaced. Hence my rather awkward...

$xml->findnodes('//*[local-name()="title"]')
[download]

i.e. "select all elements where the local part of the element's name is 'title'"

Another solution (arguably a lot more readable) would be to forcibly bind the XHTML namespace to a prefix:

$xml->documentElement->setNamespace(
  'http://www.w3.org/1999/xhtml' => 'xh',
);
[download]

And then you can freely use that prefix in XPaths.

$xml->findnodes('//xh:html/xh:head/xh:title')
[download]

This specific problem is mentioned in the XML::LibXML::Node documentation - look for the "NOTE ON NAMESPACES AND XPATH" in the documentation for the findnodes method.

perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'

Comment on Re^5: extracting data from HTML Select or Download Code

Replies are listed 'Best First'.
Re^6: extracting data from HTML by Jurassic Monk (Acolyte) on Jun 06, 2012 at 15:37 UTC
that is pretty awkward! strangely enough it seems that the other modules do not bother about the namespaces. The reasoning behind the namespaces and that those thus need to be included in the XPath is clear. Parsing the badly formed HTML into a XML-structure should ofcourse lead to proper XHTML, I do see that. Hence nsURI does use http://www.w3.org/1999/xhtml. I made a dump of the 'title' node: `'nsuri' => 'http://www.w3.org/1999/xhtml', 'suffix' => 'title', 'qname' => 'title', 'children' => [ 'extracting data from HTML' ], 'type' => 'Element', 'attributes' => [], 'prefix' => undef` [download] As the webpage of the referenced perlmonks node is just plain html with `<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">`, I will asume, somewhere the `http://www.w3.org/1999/xhtml` must have slipped in. It would be nice to override that - or, it would be nice to just remove it from each and every node in the XML-Document. Maybe there is a way to bypass those default namespaces, or do some quirks on the XPath expression, before applying it to the document All in all... yes I do desire to be fully knowlageable off all ins and outs that come with XML, XML-Schema, XPath, XSLT and more... But I don't apreciate this awkward namespace trouble Brethern monks, is there a way to get around this, other than inseting my own prefix? N.B. almost every example off XPath expressions, this is overlooked and simplyfied, which is a nasty bummer when you want to work with it	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^6: extracting data from HTML
by Jurassic Monk (Acolyte) on Jun 06, 2012 at 15:37 UTC

that is pretty awkward!

strangely enough it seems that the other modules do not bother about the namespaces.

The reasoning behind the namespaces and that those thus need to be included in the XPath is clear. Parsing the badly formed HTML into a XML-structure should ofcourse lead to proper XHTML, I do see that. Hence nsURI does use http://www.w3.org/1999/xhtml.

I made a dump of the 'title' node:

          'nsuri' => 'http://www.w3.org/1999/xhtml',
          'suffix' => 'title',
          'qname' => 'title',
          'children' => [
                          'extracting data from HTML'
                        ],
          'type' => 'Element',
          'attributes' => [],
          'prefix' => undef
[download]

As the webpage of the referenced perlmonks node is just plain html with <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">, I will asume, somewhere the http://www.w3.org/1999/xhtml must have slipped in. It would be nice to override that - or, it would be nice to just remove it from each and every node in the XML-Document.

Maybe there is a way to bypass those default namespaces, or do some quirks on the XPath expression, before applying it to the document

All in all... yes I do desire to be fully knowlageable off all ins and outs that come with XML, XML-Schema, XPath, XSLT and more...

But I don't apreciate this awkward namespace trouble

Brethern monks, is there a way to get around this, other than inseting my own prefix?

N.B. almost every example off XPath expressions, this is overlooked and simplyfied, which is a nasty bummer when you want to work with it

[reply]
[d/l]
[select]