in reply to Re: HTML-Parser: a newbie question: need to extract exactly line 999 out of 5000 files..
in thread HTML-Parser: a newbie question: need to extract exactly line 999 out of 5000 files..

Hello Morgon, good evening! Many many thanks for the quick reply!

you understood right! That is exactly what is wanted

i like your advices!

the fourth is the point where i am now! Now i have 5000 files with one line that is extracted:

you say: 4) You have now a collection of html-files consisting only of the previous line 999 that you can parse further with whatever tool you want.

how to proceed. Note i wwant to store this results in a db. Can i go and replace some html-elements with CSV !!? is this doable. Or should i parse the results with html-tokeParser!? I have little experience - but i guess that this job goes over my heard! ;-)

look forward to hear from you
  • Comment on Re^2: HTML-Parser: a newbie question: need to extract exactly line 999 out of 5000 files..

Replies are listed 'Best First'.
Re^3: HTML-Parser: a newbie question: need to extract exactly line 999 out of 5000 files..
by morgon (Priest) on Oct 16, 2010 at 01:12 UTC
    You simply have to extract the data-fields now.

    There are several ways to do it.

    As we've gotten rid of a lot of crap by only using one line of the original html, you could use a regular expression here, but that is in general not a good idea for decomposing html.

    Personally I like HTML::TreeBuilder::XPath that you would have to install from CPAN.

    Here is how you would then extract the name from one of the files with it:

    use strict; use HTML::TreeBuilder::XPath; my $tree = HTML::TreeBuilder::XPath->new; #use real file name here open(my $fh, "<", "file.html") or die $!; $tree->parse_file($fh); my ($name) = $tree->findnodes(qq{/html/body/table/tr[1]/td[2]}); print $name->as_text;
    As you can see you simply use an xpath-expression to indentify the node you want.

    So how to determine that?

    I use a Firefox-plugin called XPather, that allows you to simply click on a html-element and extract the corresponding xpath.

    So you load the file you want to parse in Firefox, click on the stuff you want, get the xpath and use that in the perl-script.

    Hope that gets you started...

      Hello dear morgon:

      many thanks for posting! Your code and your ideas are just great!! Really: I am allmost convinced!


      use strict; use HTML::TreeBuilder::XPath; my $tree = HTML::TreeBuilder::XPath->new; #use real file name here open(my $fh, "<", "file.html") or die $!; $tree->parse_file($fh); my ($name) = $tree->findnodes(qq{/html/body/table/tr[1]/td[2]}); print $name->as_text;
      i have had a look at the ideas of you morgon - that seems to be also an appropiate way. But at the moment i have issues to get the correspodending xpath-expressions: I tried to determine the corresponding xpath-expressions that needs to be filled in the Perl-programme. I need some help here: The 5000 HTML-files all do look the same!


      See one of the example sites: http://www.kultusportal-bw.de/servlet/PB/menu/1188427/index.html?COMPLETEHREF=http://www.kultus-bw.de/did_abfrage/detail.php?id=04313488

      in the grey shadowed block you see the wanted information: 17 lines that are wanted. Note - i have 5000 different HTML-files - that all are structured in the very same way!

      That means i would be happy to have a little template that can be runned with HTML::TokeParser::Simple and DBI.

      That would be great!!

      Morgon - I would be happy to hear from you

      best regards perlbeginner1

      http://www.kultusportal-bw.de/servlet/PB/menu/1188427/index.html?COMPLETEHREF=http://www.kultus-bw.de/did_abfrage/schnellsuche.php do a search with *.* -> then you get the result pages