perl_lover_girl has asked for the wisdom of the Perl Monks concerning the following question:

This node falls below the community's minimum standard of quality and will not be displayed.

Replies are listed 'Best First'.
Re: user-agent question [light one for the quick perler]
by marto (Cardinal) on Oct 01, 2006 at 10:04 UTC
    When you posted this thread you informed us that you do not have permission to parse all of these sites. So to answer your question, yes this is all possible, but without permission you should not be doing it. From your first post:

    "grab all the data out of this forum - and get it local and then after wards put it in a local database - of a phpBB-forum - is this possible"?!"? "

    You are essentially wanting to take user contributed data from various forums and copy all of it and insert it into your own forum. without being given permission from the site owners. IMHO big no no.

    Martin
Re: user-agent question [light one for the quick perler]
by shmem (Chancellor) on Oct 01, 2006 at 09:29 UTC
    Well, if that code runs (I didn't test it) with
    my $url = "http://www.mysite.com/forums/";

    it should run also with

    my $url = "http://www.kite2fly.com/forum/sitemap.php";

    If you want to run it agaist multiple URLs, just stuff them into an array and itereate over them:

    my @urls = qw ( http://www.mysite.com/forums/ http://www.kite2fly.com/forum/sitemap.php http://www.mynak.com/forum/sitemap.php http://www.dizign.de/forum/sitemap.php ); foreach my $url(@urls) { my @links; get_threads($url); foreach my $page (@links) { .... } }

    Or... what is exactly your problem?

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}