Hello dear monks good morning
I am trying to use LWP::UserAgent on the same URLs
see below with different query arguments, and i am wondering if LWP::UserAgent provides a way for us to loop through the query arguments?
I am not sure that LWP::UserAgent has a method for us to do that.
I tried to figure it out. And i digged deeper in the Manpages and Howtos: we can have
a loop constructing the URLs and use LWP::UserAgent repeatedly:
for my $id (0 .. 100000)
{
$ua->get($url."?id=21&extern_eid=".(09-$id))
//rest of the code
}
or - much
better - i do that:
for my $i (0..10000) {
$ua->get('http://www-db.sn.schule.de/index.php', id => 21, extern_ui
+d => $i);
# process reply
}
Well, alternatively we can add a request_prepare handler that computes and add the query arguments before we send out the request.
Do you think that this fits the needs? look forward to your review of this issue!
greetings
beginner
What is aimed: Here on this following site we find a list of many schools: see the page with the subsequent results
- approx more than 1000 sites
http://www-db.sn.schule.de/index.php?id=25
i want to
fetch all the sites that are listet on this page - and therefore i want to use LWP::UserAgent; - and subesquently i want to parse them.
the sites can be
reached directly - by constructing
in other words the subsites of the overview can be reached via direct links... see the following.
http://www-db.sn.schule.de/index.php?id=21&extern_eid=1543
http://www-db.sn.schule.de/index.php?id=21&extern_eid=709
http://www-db.sn.schule.de/index.php?id=21&extern_eid=789
http://www-db.sn.schule.de/index.php?id=21&extern_eid=1297
well long speech short sense: - i try this out - and let you know all the findings
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.