KarthikK has asked for the wisdom of the Perl Monks concerning the following question:

Hello Everyone, I have a tricky (for me atleast) problem 1. I have to login to a website which is secured (https) using userid and password via BASIC authentication 2. I have to get the session from the login page and use that session to call one more page to upoad a file since the login page requires authentication and it redirects to login page. 3. The upload page returns a text file which i have to download? and read the conents and do some data processing. LoginPage: https://mysite.com/login.do UploadPage: https://mysite.com/fileupload.do I have been trying using the LWP but with no luck. Also i am behind a proxy server and whenever i try to execute my script, I always get the following:

LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: POST https://mysite.com/login.do
LWP::UserAgent::_need_proxy: Proxied to http://user:pwd@proxy_server:port
HTTP::Cookies::add_cookie_header: Checking mysite.com for cookies
HTTP::Cookies::add_cookie_header: Checking .com for cookies
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: POST https://mysite.com/login.do HTTP/1.0
Host: mysite.com
Proxy-Authorization: Basic h2DrWXxyOcthqnRoaUz=
User-Agent: libwww-perl/5.53
Content-Type: application/x-www-form-urlencoded

LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.0 200 OK
Content-Type: text/html
Refresh: 0; URL=https://mysite.com/login.do

<HTML></HTML>
LWP::Protocol::http::request: HTTP/1.0 200 OK
LWP::Protocol::collect: read 15 bytes
LWP::UserAgent::request: Simple response: OK


I dont see any error but always the 200 OK message as seen above Below is the code module 
#!/usr/bin/perl
use warnings;
use strict;
use LWP::UserAgent;
use HTTP::Cookies;
use HTTP::Request::Common qw(POST);
use LWP::Debug qw(+);

my $upload_url = 'https://mysite.com/fileupload.do';
my $login_url = 'https://mysite.com/login.do';

&postHTTPS();

sub postHTTPS
{ 
    my $cookie_jar = HTTP::Cookies->new;    #HTTP::Cookies->new(autosa
+ve => 1)
    $cookie_jar->clear;    
    my $action = POST ($login_url);
    my $ua = new LWP::UserAgent(keep_alive=>1,env_proxy => 0);
    $ua->proxy(['http','https'] => 'http://user:pwd@proxy_server:port'
+);
    $ua->env_proxy();
    $ua->cookie_jar($cookie_jar);  
    my $request  = new HTTP::Request ("GET" => $login_url ); 
    my $response = $ua->request($action);
    if ($response->is_error())
    {
        printf " %s\n", $response->status_line;
        print "https request error!\n";
    }else {
        my $content = $response->content();        
    }
    if ( $response->is_success ) {
     #   print $response->as_string;        
    }else {
        #print $response->status_line;
        print "\nFailure!\n";
    }
}

[download]
The above code is only to login. I am not sure how to get the session cookie from that login page and pass to the download page (code not written yet) I had also asked similar question at http://www.perlmonks.com/index.pl?node_id=645153 i got some responses but still I have a problem Is this scenario achievable in PERL? I am now learning this LWP and related modules since i am new to these Can you one help me in this regard? Thanks a lot in advance Regards, Karthik

Replies are listed 'Best First'.
Re: How to authenticate and upload file to a secured server
by Corion (Patriarch) on Oct 18, 2007 at 16:03 UTC
[reply]
Re: How to authenticate and upload file to a secured server
by ikegami (Patriarch) on Oct 18, 2007 at 16:49 UTC

    "Basic Authentication" has no relation to "BASIC" or even "Visual Basic".

    The website doesn't use Basic Authentication. It (presumably) uses HTML forms and a session cookie.

    Your proxy uses Basic Authentication, and that appears to work fine.

    You never attempt to login (i.e. you never submit the populated login form).
    I was confused by the useless my $request  = new HTTP::Request ("GET" => $login_url );
    You do do a POST, but you don't provide any of the form fields. See the examples in HTTP::Request::Common.

    Once you login, the session id will most likely be returned in a cookie. Your cookie jar code already handles that. Just use the same $ua for future requests requiring the session cookie.

[reply]
[d/l]
[select]
      Hi thanks for the reply.
      1. The website uses BasicAuthentication and not a form based authentication.
      2. By Basic authentication i meant the login box that comes in IE or other browsers where we provide user name/password/domain
      3.I have to first login using the login.do page with the POST action. Since I don't have any form variables in that page i am not posting any username/pwd
      4. I even wrote the cookie to a file but the file does not contain any data. it is empty
      i am not able to install WWW::Mechanize Module using nmake. I copy pasted the PM file and other related modules to C:\perl\lib\WWW\Mechanize folder and Mechanize.pm to WWW folder.
      also i use Windows OS and not linux.

      Thanks and regards
      Karthik
[reply]

        If it uses Basic Authentication, you'll need to call $ua->credentials, or create and use a subclass of LWP::UserAgent that overrides get_basic_credentials.

        By definition, if Basic Authentication was used, you'd get a 401 response including a "WWW-Authenticate: Basic realm="..."" header. Your site either doesn't use Basic Authentication, or you're not accessing the right page.

        Basic Authentication doesn't have any domain component. Just realm, user and password. Realm is set by the server and is not editable. I've seen the dialog to which you refer in the past, and it's not Basic Authentication. I don't know what it is.

        Look at the headers in your trace. You never got any cookie. That's why the file is empty. Basic Authentication doesn't use cookies anyway.

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom