Re: Xpather running against a simple HTML-document

My earlier example that you reuse here uses an xpath that works against the transformed html (using only line 999 - remember).

If you want to parse the original html of course you have to use a different xpath.

Which path to use is easy enough to determine with XPather (which is NOT a perl-module but a firefox-plugin): Right-click on what you want - select "Show in XPather".

I won't write the whole script for you but if you want you can hire me...

Comment on Re: Xpather running against a simple HTML-document - testing and evaluation

Replies are listed 'Best First'.
Re^2: Xpather running against a simple HTML-document - testing and evaluation by Perlbeginner1 (Scribe) on Oct 16, 2010 at 19:32 UTC
Good evening morgon and ww. and also hello dear Perl-Monks! great to hear from you! as parsing this one here: http://www.kultusportal-bw.de/servlet/PB/menu/1188427/index.html?COMPLETEHREF=http://www.kultus-bw.de/did_abfrage/detail.php?id=04313488 i now make use of use some HTML::TableExtract magic: And here i make usage of some circumscances that have to do with the color... conclusion: The Parser-job can be considered to be done yet! Now i have to work on the saving in the database! see the code that does all the fuzz `#!/usr/bin/perl use strict; use warnings; use HTML::TableExtract; use YAML; my $te = HTML::TableExtract->new( attribs => { border => 0, bgcolor => '#EFEFEF', leftmargin => 15, topmargin => 5, }); $te->parse_file('kultus-bw.html'); my ($table) = $te->tables; for my $row ( $table->rows ) { cleanup(@$row); print "@$row\n"; } sub cleanup { for ( @_ ) { s/\s+//; s/[\xa0 ]+\z//; s/\s+/ /g; } }` [download] see the Output: Schul-/Behördenname: Abendgymnasium Ostwürttemberg Schulart: Privatschule (04313488) Hausadressse: Friedrichstr.70, 73430 Aalen Postfachadresse: Keine Angabe Telefon: 07361/680040 Fax: 07361/680040 E-Mail: Keine Angabe Internet: www.abendgymnasium-ostwuerttemberg.de ÜbergeordneteDienststelle: Regierungspräsidium Stuttgart Abteilung 7 Schule und Bildung Schulleitung: Keine Angabe Stellv.Schulleitung: Keine Angabe AnzahlSchüler: 259 AnzahlKlassen: 8 AnzahlLehrer: Keine Angabe Kreis: Ostalbkreis Schulträger: <Verband/Verein> (Verband/Verein) See the HTML::TableExtract that does the magic! Also, the documentation for HTML::TableExtract is available. It does what it says it does: Extracts specific tables from HTML source code. And it does that really well. BTW: i want (need to do this with another table/site: see this page: http://www.schulministerium.nrw.de/BP/SchuleSuchen?action=672.8924536341191 Note: click all checkbuttons at the bottom of the site: Then you see a result-page with more than 6400 school-results: see at the right of the site Weitere Informationen anzeigen you can get detailed information if you click Weitere Informationen anzeigen 9 (or ten lines) Schuldaten. Schulnummer: Amtliche Bezeichnung: Strasse: Plz und Ort: Telefon: Fax: E-Mail-Adresse: Schuldaten ändern] :(this is UTF8 encoded or what) Schülergesamtzahl (this is UTF8 encoded or what) Question: can the HTML::TableExtract can be applied here to!? at the resultpage of more than 6400 shools: http://www.schulministerium.nrw.de/BP/SchuleSuchen?action=672.8924536341191 i guess so!	[reply] [d/l]

Replies are listed 'Best First'.

Re^2: Xpather running against a simple HTML-document - testing and evaluation
by Perlbeginner1 (Scribe) on Oct 16, 2010 at 19:32 UTC

conclusion:

code that does all the fuzz

#!/usr/bin/perl

use strict; use warnings;
use HTML::TableExtract;
use YAML;

my $te = HTML::TableExtract->new( attribs => {
    border => 0,
    bgcolor => '#EFEFEF',
    leftmargin => 15,
    topmargin => 5,
});

$te->parse_file('kultus-bw.html');
my ($table) = $te->tables;

for my $row ( $table->rows ) {
    cleanup(@$row);
    print "@$row\n";
}

sub cleanup {
    for ( @_ ) {
        s/\s+//;
        s/[\xa0 ]+\z//;
        s/\s+/ /g;
    }
}
[download]

Output:

HTML::TableExtract

BTW:

more than 6400 school-results:

Weitere Informationen anzeigen

HTML::TableExtract

[reply]
[d/l]