You will have to do a submit with a Suchbegriff: of "*"
That will result in a page of 5081 results. To get the sub-pages pages you want, you will have to "click" via LWP or whatever to follow these links all 5081 of them.

Start with trying to submit the search term of "*" on the main page and see if you can do that.

Hello Marshall

many thanks for the reply! i can do as you adviced. I can see the 5081 results.

Now i have to get the sub-pages pages. I have to "click" via LWP to follow these links - all 5081 of them.
And then i have to do the Job with HTML-TREEBuilder or use HTML::TokeParser!
I for one prefer HTML::TokeParser since i know this a little bit.

i have very very little experience with HTML::TokeParser (not too much - so i guess that the parser-part will be something over my skills)


but as first things come first how should the LWP-Part look!?

any and all help will be greatly appreciated!

perlbeginner1

Here is an example which uses WWW::Mechanize to visit the page, populate the field and submit the form. Error checking is left as a exercise for you, this is a short example to get you started:

#!/usr/bin/perl

use strict;
use warnings;

use WWW::Mechanize;

my $url = 'http://www.kultusportal-bw.de/servlet/PB/menu/1188427_pfhan
+dler_yno/index.html';

my $mech = WWW::Mechanize->new();
$mech->get( $url );

$mech->field('einfache_suche','*');
$mech->submit();
# $mech->content now contains the results page.
[download]

I can't read German, so you'd better check that you're not breaking any site policy regarding automation.

I would go with marto's advice about WWW::Mechanize. I haven't used it yet, but I hear that it is great. I suspect that you will find it easier to use than any advice I could give about decoding the raw HTML to get the next pages to "click" on. You are getting about 5K pages from a huge government website that performs very well. I wouldn't worry too much about fancy error recovery with retries unless you are going to run this program often.

Update:
You can of course parse the HTML content of the search results with regex, but this is a mess...

my (@hrefs) = $mech->content =~ m|COMPLETEHREF=http://www.kultus-bw.de
+/did_abfrage/detail.php\?id=\d+|g;

print "$_\n" foreach @hrefs;    #there are 5081 of these

#these COMPLETEHREF's can be appended to a main url like this:

my $example_url = 'http://www.kultusportal-bw.de/servlet/PB/menu/11884
+27/index.html?COMPLETEHREF=http://www.kultus-bw.de/did_abfrage/detail
+.php?id=04146900';
[download]

Then things get hairy and you will want to whip out some of that HTML parser voo-doo to parse the resulting table. Also, the character codings aren't consistent, for example the page has ä, but not ü which is coded as ü

I just want to adwise you to check if you are allowed to use your script on a government website. Just because it's possible it does not always mean it's a good idea. Be sensitive about data privacy when automating web things. When using the swiss army chanisaw, be aware you might cut your leg off if you're careless.

BTW: I don't feel well with people posting valid mail adresses and phone numbers instead of example data ... because I hate spam.

Did you read Choosing a username? Do you really want to stay Perlbeginner1 whole life? Just for curiosity ;-)

Hi there Brimborium

thx for sharing you ideas!

since i am a teacher and since i am working in the field of education for years i know very well what i do! I have no troubles with parsing this govermental site!

The data i am trying to get are readable - so i mechanize this reading...


BTW: one word regarding the data: These data are offical Adress-data - names and numbers of shools - nothing else.

some general adress-sets that contain nothing really sensitive!

but again - thx for sharing your ideas. BTW: what is wrong with my username; i am a beginner.

regards - perl beginner1

There is nothing wrong with your name, but you will stay a beginner forever, at least with your name...

I just want to point out to choose the right way to do things to avoid causing more damage than benefit. You can use a club to get a fly away from your friends shoulder, but he may not recover from your favour. You have a powerful tool with perl, I just want to be sure that you use it wisely.

Reading a lot of files in a short time from a public server could be misinterpreted... if you are a teacher you should be aware of the consequences ... you might kill the server with a buggy script. I just want to predict you from having to say "Oh, I did not WANT that, it was really not my intention"

A phonebook is also available to public, but I dislike the idea of having it machine redable for dialing bots ...

I'm a software developer since many years and if you're using example or test code on real data on a public server, you really do not know what you do form my point of view.