Vanquish has asked for the wisdom of the Perl Monks concerning the following question:

I am kinda new to Perl but i am trying, what basically i want to do is login to IITS system and go to particular page parse it and get webpage content.Though i havent reached to the point for parsing i want to go step by step. But I get Response status is: 405 Method not allowed error. Help Needed
#!/usr/bin/perl use LWP::UserAgent; use HTTP::Cookies; use HTTP::Request::Common ; #!/usr/bin/perl use LWP::UserAgent; use HTTP::Cookies; use HTTP::Request::Common ; use HTTP::Request; use HTTP::Response; use HTTP::Headers; use HTML::Parser; use XML::Parser; my $login = { username => 'guestuser', userPassword => 'temp', ip_address => '10.70.30.20', successURL => 'home.asp', submit => 'login', }; my $ua = LWP::UserAgent->new( keep_alive => 1 ); timeout => 30 ); my $cookie_jar = HTTP::Cookies->new; $cookie_jar->add_cookie_header($request); $ua->cookie_jar( $cookie_jar ); my $header = HTTP::Headers->new( Content_Type => 'text/html' +, User_Agent => 'Release_Bu +ild/1.0' ); $header->as_string; my $request = HTTP::Request::Common::POST( 'http://10.1.19.22', $l +ogin, User_Agent => 'Relea +se_Build/1.0' ); $request->as_string; my $response = $ua->request( $request ); print $response->as_string(); $cookie_jar->extract_cookies($response); $cookie_jar->as_string(); while ($response->is_redirect) { my $location = $response->header( "location" ); $request = HTTP::Request->new( "GET", "http://10.1.19.22/" . $ +location, $header ); print $request->as_string; $response = $ua->request( $request ); print $response->as_string(); } unless ($response->is_success) { print "Response status is: ", $response->status_line(); return undef; }

Replies are listed 'Best First'.
Re: Webpage Contents
by Elijah (Hermit) on Jun 29, 2004 at 17:05 UTC
    Now I am not sure of your implementation but something as simple as the following will grab the site content for you:
    #!perl -w use strict; use LWP::Simple; my $site = $ARGV[0] || "http://www.perlskripts.com"; my $command = get($site); die "An error has occured!" unless defined $command; print $command;
    From there you can do whatever you want with it as far as parsing goes. I even added some interaction from the user functionality in the above code which is not needed. At the very least you can use this code to see if the site has an issue or your code has an issue. I ran this against my own site and perlmonks site and had no issues.

    You also seem to have a minor code error that keeps the script from running. Change the following:

    my $ua = LWP::UserAgent->new( keep_alive => 1 ); timeout => 30 );
    to this:
    my $ua = LWP::UserAgent->new( keep_alive => 1, timeout => 30 );
    Also change the following:
    unless ($response->is_success) { print "Response status is: ", $response->status_line(); return undef; }
    to:
    print "Response status is: ", $response->status_line() unless ($respo +nse->is_success);
    The return is not needed nor is it allowed.

    Now with both those minor changes in place I run the script and get an error code 500 which I am assuming is because you have changed the IP addresses in your code to protect the innocent. :-)

    If you plug in all your correct data back into this script you might have better results.


    www.perlskripts.com
Re: Webpage Contents
by Grygonos (Chaplain) on Jun 29, 2004 at 16:18 UTC
    Welcome to the monastery, {if you haven't already been :)}

    While I have no experience coding LWP it would seem to me that either the POST or GET method is not being allowed on this server. Also you may want to state explicitly which line causes the error, so you could see which method it was not taking a liking to.

    Have you tried this on a website you own (assuming this one is not run by you)? even one setup on your local machine via Apache/IIS would be sufficient.. can you pull data there? My first step of debugging (with no LWP exp as stated) would be to see if I or the site in question is the problem. Try it on a site of your own to see if you have a malformed request. If it works there, then it may be something with the site you are requesting from

    Just my thoughts,