Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

logging in to secure site

by dannoura (Pilgrim)
on Apr 30, 2004 at 14:46 UTC ( [id://349412]=perlquestion: print w/replies, xml ) Need Help??

dannoura has asked for the wisdom of the Perl Monks concerning the following question:

hi,

I'm trying to login to a this site, which uses https. Problem is, I get an internal server error. Here's my code:

#! c:\perl\bin -w use strict; use LWP::UserAgent; use HTTP::Request::Common; use HTTP::Cookies; use LWP::Debug qw(+); use LWP::Protocol::https; use crypt::SSLeay; my $ua = LWP::UserAgent->new; $ua->cookie_jar(HTTP::Cookies->new(file => 'cookie_jar', autosave =>1) +); # configure LWP::UserAgent to follow redirects after POST push @{ $ua->requests_redirectable }, 'POST'; my $request=$ua->request(POST "http://www.saxobank.com/", { username =>'me', userpass =>'secret', submit =>'Submit' }); print $request->is_success ? $request->content : "failed\n";

The output from LWP::Debug is:

LWP::UserAgent::new: () LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.saxobank.com for cookie +s HTTP::Cookies::add_cookie_header: Checking .saxobank.com for cookies HTTP::Cookies::add_cookie_header: Checking saxobank.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST http://www.saxobank.com/ LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::UserAgent::request: Simple response: Internal Server Error

Any ideas?

Update:Using the above code with the URL https://www.saxobank.com/Default.aspx?ID=867 gives the response:

LWP::UserAgent::new: () LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.saxobank.com for cookie +s HTTP::Cookies::add_cookie_header: Checking .saxobank.com for cookies HTTP::Cookies::add_cookie_header: Checking saxobank.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST https://www.saxobank.com/Default.as +px?ID=867 LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 47 bytes LWP::Protocol::collect: read 100 bytes HTTP::Cookies::extract_cookies: Set cookie ASP.NET_SessionId => 3v0qpi +550pgfofqpbbv3qc45 LWP::UserAgent::request: Simple response: Found LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.saxobank.com for cookie +s HTTP::Cookies::add_cookie_header: - checking cookie path=/ HTTP::Cookies::add_cookie_header: - checking cookie ASP.NET_SessionId +=3v0qpi550pgfofqpbbv3qc45 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: Checking .saxobank.com for cookies HTTP::Cookies::add_cookie_header: Checking saxobank.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST http://www.saxobank.com?ID=867 LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::UserAgent::request: Simple response: Internal Server Error

Which leads me to believe (because two requests were sent, with only the second one being unsuccessful) that a redirection is supposed to occur, but which LWP does not follow, despite the line specifically instructing it to do so. No solution as yet...

Replies are listed 'Best First'.
Re: logging in to secure site
by matija (Priest) on Apr 30, 2004 at 15:17 UTC
    Any ideas?
    Yeah: The submit button has an onclick javascript, which changes the form's action to https://www.saxobank.com/Default.aspx/?id=2&Lan=EN&Au=0&Grp=0

    At the very least, you will need change the URL you're submitting to and add the id, Lan, Au and GRP to the parameters when you log in. There may still be other Javascript hocus-pocus that you need to unravel, though.

      Thanks for your help. I tried submitting https://www.saxobank.com/Default.aspx/?id=2&Lan=EN&Au=0&Grp=0 in the POST method. This gave me another page with username and password fields in plain html (below). Submitting to that with the URL https://www.saxobank.com/ didn't work. Excuse my ignorance, but I can't figure out how to submit using the information from the javascript. Can you figure it out?

      <td class="qblock-head" colSpan="2">Member Login (Encrypted)</td> </tr> <tr> <td class="qst">User ID &nbsp;</td> <td class="qst"><input class=input type=text m +axLength=50 size=20 value="" name="txtUID" id="txtUID"> </td> </tr> <tr> <td class="qst">Password &nbsp;</td> <td class="qst"><input class="input" type="pas +sword" maxLength="50" size="20" name="txtPWD" id="txtPWD"> </td>
        Your form fields are named incorrectly. From the HTML above, you'll need to supply the fields "txtUID" and "txtPWD" for your username as password, respectively. In addition, you also need the hidden form field "__VIEWSTATE", and the "submit" field (named for the submit button). In the code below, I also updated the URL to match that set by the javascript on the web page.
        my $request=$ua->request( POST "https://www.saxobank.com/Default.aspx/?id=2&Lan=EN&Au=0&Grp=0" +, { __VIEWSTATE => 'dDwxOTk0Mzg2NjQzOzs+/+MSvllHQREYBFP5zZXrPV/rhdM=', txtUID => 'me', txtPWD => 'secret', submit => 'log in' });
        It doesn't appear that the "__VIEWSTATE" field has a unique value for each page load, but if it did, you could modify your code to load the home page to retrieve the value for the "__VIEWSTATE" field:
        my $viewstate= get_viewstate( $ua ) or die "can't get viewstate field" +; + my $request=$ua->request( POST "https://www.saxobank.com/Default.aspx/?id=2&Lan=EN&Au=0&Grp=0" +, { __VIEWSTATE => $viewstate, txtUID => 'me', txtPWD => 'secret', submit => 'log in' }); + print $request->is_success ? $request->content : "failed\n"; + # retrieve value for hidden __VIEWSTATE field (unique for each browser + load?) sub get_viewstate { my $ua= shift; + require HTML::TokeParser; my $request= $ua->request(GET "http://www.saxobank.com/"); die $request->status_line unless $request->is_success; + my $p= HTML::TokeParser->new( \$request->content ); + while( my $tag= $p->get_tag("input") ) { return $tag->[1]{value} if $tag->[1]{name} eq '__VIEWSTATE'; } }
        --sacked
        I might have been able to if you had submitted enough information, but you didn't so I can't.

        In particular, you didn't include the <form> tag, which might tell us the right URL, nor the <submit> tag that would show us if they were playing any further games with javascript.

        Before you come back here, look at the form tag, and try to submit to the URL listed in the action field.

Re: logging in to secure site
by nmcfarl (Pilgrim) on Apr 30, 2004 at 15:26 UTC

    Well, since an internal server error happens on the server, it usually means the script the page was submitted to did not receive all the information it was expecting (and it has poor error handling). Often this is a cookie or state variable problem, but in this case I think it is a javascript issue.

    The login button on the saxobank page runs some javascript, but LWP does not interpret javascript, so the javascript doesn't do it's work. I would guess this javascript is vital to the site, and the cause of your error.

    The javascript looks like it is just changing the form action location, so you should be able to fake it out with some work on the perl side.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://349412]
Approved by allolex
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (2)
As of 2024-04-24 17:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found