I'm trying to simulate a login to a web page through a perl script. I'm using LWP::UserAgent, HTTP::Cookies, and HTTP::Request. The problem is that the login for the web site occurs in two phases (two separate web pages). In the first phase you enter in your username and password info and the second phase you select where you want to be redirected. I can get the redirection page just fine (in other words, my perl script handles the login and password page just fine - also I can get to other restricted pages on the site so I know that the perl script is handling the login page correctly).

The problem with the second web page (where the user selects where he wants to be redirected) is that there are about 30 hidden form fields on the page that are submitted when you click the submit button. My perl script needs to be able to pass these hidden form fields and there values to the server. Is there any efficient way (using CGI.pm or some module like that) that would grab all form data and allow me to just pass it through?

Thanks.

In reply to Parsing Web Page by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.