gr0f has asked for the wisdom of the Perl Monks concerning the following question:

Hello fellow Perl programmers...

I'm trying to develop a script that goes out onto the Internet and automatically enters information into a sites database. Problem is, the site is protected by the standard HTML text username and password boxes who's values are sent to a script. I can enter these values in the URL of initial HTTP request, but of course the server forgets who I am during the next transaction. I'm not really sure if using the: 

$ua->credentials
argument would solve the problem or not, but my initial attemps were unsuccessful. I'm wondering what environmental variables may contain the password/username combo and how I might be able to change them. A web browser obviously caches something that it sends to the server each time since it isn't plagued by this problem. I need to know what that might be a how to do it with LWP...

Thanks, Tim...

Replies are listed 'Best First'.
Re: LWP Authorization Problem
by twerq (Deacon) on Jul 24, 2001 at 21:47 UTC
    (from comp.lang.perl.modules)

    As usual, there's more than one way to do it.

    1. Use the credentials method of the LWP::UserAgent object:

         $ua->credentials($netloc, $realm, $uname, $pass)

    [download]

    Eg: Say your URL is http://www.somesite.com/page.html and when you try this with your browser, it pops up a box saying 'Enter username and password for XYZ'.

    $netloc in this example is 'www.somesite.com' and $realm in this case is 'XYZ'.

    2. Call the authorization_basic method of your HTTP::Request object before passing it to the LWP::UserAgent's request method:

       $request->authorization_basic($uname, $pass);

    [download]

    If you use method 1, the LWP::UserAgent will send one request which the server will reject with a 401 error. One of the headers in the 401 response will be the realm. The LWP::UserAgent will then look up it's internal table (populated by your calls to credentials) to find a matching netloc and realm. If this lookup returns a username and password, the LWP::UserAgent will resend the request with a basic authorization header.

    The second method allows you to set the username and password before making the initial request - assuming you supply the right credentials there will be no 401 error. There is no need to specify netloc and realm, as they are only used by the LWP::UserAgent's lookup routine.

    'perldoc LWP::UserAgent' will tell you more about 1

    'perldoc HTTP::Message' will tell you more about 2 (HTTP::Request is derived from HTTP::Message).


    --twerq
[reply]
[d/l]
[select]
      Unfortunately it doesn't pop up a box, the fields are just embeded in the HTML like logging into hotmail for example. I've set up domain passwords on Apache before but is this the same type of authorization problem, or something different? As I first mentioned, the password and login can be provided if they are included in the URL like variables using the GET method.

      Here's the login page so you can see what I mean:
      http://www.environet.gov.on.ca/

      And thanks for the first timely reply, I didn't think I'd get something so fast...

      -Timbo

[reply]
        Are you handling any cookies it throws back at you? It may be using these for Auth?

        Have you used a packet monitor to watch what goes down the wire using a real browser??

        --
        RatArsed

[reply]
Re: LWP Authorization Problem
by twerq (Deacon) on Jul 26, 2001 at 08:24 UTC
    I guess then, in that case (since the page doesn't use HTTP authentication -- my mistake). . .

    You're going to have to reverse-engineer their webpage a little bit. It's usually only as hard as finding out the <FORM> values of the two input boxes it uses for username and password.

    Assuming these two values are named Username and Password, we can either use HTTP::Request POST or HTTP::Request GET to send that page our values.

    Play around with this (untested): 
    
   # method 1:
   my $req = new HTTP::Request POST => 
      'http://www.environet.gov.on.ca/dwws-app/DWWSApp';
   $req->content_type('application/x-www-form-urlencoded');
   $req->content('username=frankie&password=tomato');

   # method 2:
   $req = HTTP::Request->new(GET => 
      'http://www.environet.gov.on.ca/dwws-app/DWWSApp?username=franki
+e&password=tomato');

    [download]

    . . . where dwws-app/DWWSApp is the name of the form in the ACTION attribute of the <FORM> element.

    . .. post back with your findings!

    --twerq
[reply]
[d/l]
Re: LWP Authorization Problem
by Anonymous Monk on Aug 06, 2001 at 16:14 UTC
    I have no difficulty passing a username/passwd combination and reading an unencrypted page. When I use the same method with an SSL encrypted page, I am unable. Is there something besides authorization_basic that I should be using? TIA.
[reply]
      That is a whole other can of worms (SSL).

      You need to be really explicit and detailed when asking questions, too much guess work is going on as to what you are trying to do.

      please compile a list of things 1. you've tried,
      2. you want to accomplish,
      3. you think is going on.

      You're using a webform (check)
      Are you using cookies?
      The page is viewed via SSL, ie its encrypted? (?you tell me?)
      You need to look here

[reply]

Back to Seekers of Perl Wisdom