Re^4: problem HTML::FormatText::WithLinks::AndTables

Hello:

Thanks for that example. It gave me something to thing about.

The first thing I noticed is that you are using:
-> querySelectorAll('table#game_info tr') # get all rows from +game_info table
to find the game table. Not all website provide an id on the table
please consider the following webpage:
http://www.databasefootball.com/boxscores/scheduleyear.htm?yr=1985&lg=nfl
What I am want to extract from here is the basic game info week by week.
When I look at the source for the page I do not see an id.
Anyway I wrote this bit of code:

use strict;
use warnings;

use HTML::HTML5::Parser;
use XML::LibXML::QuerySelector;
use XML::LibXML;
use Data::Dumper;


my $url = "http://www.databasefootball.com/boxscores/scheduleyear.htm?
+yr=1985&lg=nfl";

 my $parser =  HTML::HTML5::Parser->new;
 my $doc  =  $parser->parse_file($url);


print Dumper $doc;
print $doc->toString;
[download]

just to see what HTML::HTML5::Parser would do with the databasefootbal.com season page.
I expected the webpage would get parse into some sort of XML structure I could query. But I do not see that.
Hope I am not testing your patience, but how would I get at the tables of scores by week in the above webpage.
Many thanks

KD

Comment on Re^4: problem HTML::FormatText::WithLinks::AndTables Download Code

Replies are listed 'Best First'.
Re^5: problem HTML::FormatText::WithLinks::AndTables by tobyink (Canon) on Mar 12, 2013 at 08:02 UTC
`Dumper` is not especially useful for inspecting XML::LibXML's objects. You see, XML::LibXML is a wrapper for a C library (libxml2) and the real guts of the objects live within the C library. This has frustrated me in the past too. The HTML5 spec differentiates between two kinds of `xml:lang` attributes (attributes called `lang` in the `xml` namespace, versus attributes called `xml:lang` in no namespace!) and `toString` doesn't distinguish between those. So this was tricky to debug when working on HTML::HTML5::Parser. I wrote XML::LibXML::Debugging as a solution, though I rarely use it these days, and don't give it much attention maintenance-wise. The following example gives you a big Perlish tree of nested hashes and arrays: `use strict; use warnings; use Data::Dumper; use HTML::HTML5::Parser; use XML::LibXML::Debugging; my $document = HTML::HTML5::Parser->load_html(IO => \*DATA); print Dumper( $document->toDebuggingHash ); __DATA__ <!doctype html> <title lang="en">Example</title> <table><tr><td xml:lang="en">Hello world</table>` [download] However the best ways of "navigating" the XML tree are to use `querySelector`/`querySelectorAll` provided by XML::LibXML::QuerySelector (which allow you to choose elements using CSS selectors) or if you need something more powerful, using XML::LibXML's built in XPath support. You don't always need an `id` attribute to select the data you want. For example, say you want to select the third `<table>` on a page, you could just do: `my @all_tables = $document->querySelectorAll('table'); my $wanted_table = $table[2];` [download] Or to select the first `<table>` within `<div class="foo">`: `my @all_tables = $document->querySelectorAll('div.foo table'); my $wanted_table = $table[0];` [download] Or, because `querySelector` returns the first match, this is the same: `my $wanted_table = $document->querySelector('div.foo table');` [download] `package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name`	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^5: problem HTML::FormatText::WithLinks::AndTables
by tobyink (Canon) on Mar 12, 2013 at 08:02 UTC

Dumper is not especially useful for inspecting XML::LibXML's objects. You see, XML::LibXML is a wrapper for a C library (libxml2) and the real guts of the objects live within the C library.

This has frustrated me in the past too. The HTML5 spec differentiates between two kinds of xml:lang attributes (attributes called lang in the xml namespace, versus attributes called xml:lang in no namespace!) and toString doesn't distinguish between those. So this was tricky to debug when working on HTML::HTML5::Parser.

I wrote XML::LibXML::Debugging as a solution, though I rarely use it these days, and don't give it much attention maintenance-wise. The following example gives you a big Perlish tree of nested hashes and arrays:

use strict;
use warnings;

use Data::Dumper;
use HTML::HTML5::Parser;
use XML::LibXML::Debugging;

my $document = HTML::HTML5::Parser->load_html(IO => \*DATA);
print Dumper( $document->toDebuggingHash );

__DATA__
<!doctype html>
<title lang="en">Example</title>
<table><tr><td xml:lang="en">Hello world</table>
[download]

However the best ways of "navigating" the XML tree are to use querySelector/querySelectorAll provided by XML::LibXML::QuerySelector (which allow you to choose elements using CSS selectors) or if you need something more powerful, using XML::LibXML's built in XPath support.

You don't always need an id attribute to select the data you want. For example, say you want to select the third <table> on a page, you could just do:

my @all_tables   = $document->querySelectorAll('table');
my $wanted_table = $table[2];
[download]

Or to select the first <table> within <div class="foo">:

my @all_tables   = $document->querySelectorAll('div.foo table');
my $wanted_table = $table[0];
[download]

Or, because querySelector returns the first match, this is the same:

my $wanted_table = $document->querySelector('div.foo table');
[download]

package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name

[reply]
[d/l]
[select]