Login into a page and getting a response from the web site

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks, I have an application in perl that can be divided into two parts:
1. Connect to a website(http://www.xyz.com/login.aspx) through perl script in Unix. The website should have a logic to connect to the website by passing user name(say:- user) and password(say:- pass). Once the credentials are supplied, HTML response of another page (say page1.aspx; url: http://www.xyz.com/Protected/page1.aspx) should be returned.

2. Now, get a number from Oracle table and enter this number into page1.aspx and press the "Search" button in page1.aspx to get HTML response of page2.aspx(url: http://www.xyz.com/Protected/page2.aspx) and store this HTML response of page2.aspx in a file for further processing.

I have tried to get the HTML response for page1.aspx but everytime I am getting the response from the login.aspx page only. I have used LWP::Simple and LWP::UserAgent modules till now for this application.
How can I go about to do these two steps? What other modules can I use?
Thanks in advance

Comment on Login into a page and getting a response from the web site

Replies are listed 'Best First'.
Re: Login into a page and getting a response from the web site by moot (Chaplain) on Mar 30, 2005 at 05:24 UTC
WWW::Mechanize makes filling in and submitting web forms very easy.	[reply]
Re: Login into a page and getting a response from the web site by kprasanna_79 (Hermit) on Mar 30, 2005 at 05:39 UTC
Hai For this kind of process try WWW::Mechanize U can try HTTP::Cookies if u need cookie support to ur crawling web site. See also perl monks search results --prasanna.k	[reply]
Re: Login into a page and getting a response from the web site by InfiniteLoop (Hermit) on Mar 30, 2005 at 05:26 UTC
LWP::UserAgent has redirect_ok(), "to determine whether a redirection in the request should be followed.", check it out.	[reply]