I am writing a web client script that implements the
following scenario that is done using a browser:
1. set url to a sites login page (say to http://www.someplace.com/login.asp
for example)
2. Type in a valid email address and password in a form
3. click the submit button
4. the browser goes to the "home" page which means I'm logged in
(in this example http://www.someplace.com/home.asp )
5. once here I do other stuff - more forms filled out and submit buttons
clicked, etc.

So far the script only manages to to get to step 3.2
(not even halfway to step 4!). The login works (or seems to).

The steps in the script are basically this:

1. get a UserAgent object
2. Create a request object with the POST method and the url.
3. make a request with the agent and get a response
4. parse the content and get a form object
5. set the email and password in the form object with the valid values
6. create a new request by "clicking" the submit button like this:
my $new_req = $form->click;
7. have the agent make another request using the request created in step 6
and get the response
8.PROBLEM IS HERE - now I want to go the the http://www.someplace.com/home.asp
url, and try to do so
9. create a third request object using the above url (I also tried setting
the url using $response->header('location') and it does not work either)
with POST
10. have the agent send the request and get the response.
11. end result a code of 302, the "Object moved" title, and an
href="home.asp", not the page I wanted!

On reading the available doc (or at least what I could find)
my understanding is that each resquest/response pair are stateless.
Does this mean that the server is firing up a new instance of the
cgi script for each request and when the response comes back,
the cgi script ends? If so, the connection is broken.
How can I acheive persistence such that I can keep the same
instance of the cgi script alive while MY web client script is alive?
This way, it will remember that I'm already logged in. (I'd love a module
called LWP::UserAgent::Persistent !!) Am I thinking of this problem the right way?
I'm new to using modules in perl, and not very experienced
with web programming. I'm comfortable enough with perl to parse out the data I
want once I can get the right page coming back to me.

Am I not handling the redirection correctly? I know I want to follow
the link "home.asp" that is returned (this is how the browser works).
Why doesn't the server send me the same page as the browser when I log in?

Any help appreciated, I'm basically stuck here and don't know what
to try next. This MUST be possible!

My apologies if you have seen this question before.

Thanks
Casey

In reply to Getting past login screen in web client by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.