What's the easiest way to have perl fill in a web form & submit it?

jdlev has asked for the wisdom of the Perl Monks concerning the following question:

I've got what I hope is a simple question. I have to have a program go to a website, and fill in a web form, and from there submit the update. It "may" also have to go through a gateway page, and enter a username & password in order to access this page.

Here's what I'd like to start with. Simply have the program go to a web page, update a field, and hit submit. Done.

Next comes the hard part. The full program goes to the website login page. Enters username/password. Clicks "Submit". Then finds a link on the page and clicks on the link. Once it clicks on the link, it will be at the form update page, and can submit the updated form information.

Any help is greatly appreciated!

I love it when a program comes together - jdhannibal

Comment on What's the easiest way to have perl fill in a web form & submit it?

Replies are listed 'Best First'.
Re: What's the easiest way to have perl fill in a web form & submit it? by kennethk (Abbot) on Feb 04, 2010 at 22:51 UTC
All this and more is possible, using the fabulous CPAN module WWW::Mechanize. You will have to look at the target page sources to program the field names, etc. but it's all pretty straight forward. Examples are available at WWW::Mechanize::Examples and we can certainly help you debug once you've started.	[reply]
Re^2: What's the easiest way to have perl fill in a web form & submit it? by jdlev (Scribe) on Feb 08, 2010 at 20:37 UTC
Thanks for the help! Mechanize looks like a great pm! I've hit a big snag I think. I can't get Mech to recognize the form. In the web page source code, there is no form name. They are using javascript to verify the data on the form for the login page. Perl can't find the form for whatever reason. So what are my options? Also, what's an easy way to see if the program successfully made it past the login page and got to the next page? I love it when a program comes together - jdhannibal	[reply]
Re^3: What's the easiest way to have perl fill in a web form & submit it? by kennethk (Abbot) on Feb 08, 2010 at 23:57 UTC
If the web page in question relies heavily on JavaScript, then you can't directly use Mechanize as it doesn't include a JavaScript engine - see WWW::Mechanize::FAQ. There are a couple of alternatives available. You might be able to use Win32::IE::Mechanize or Mozilla::Mechanize, which are supposed to offer external control over your browser. I've never used either, though it seems Win32::IE::Mechanize is better developed and tested. A more roll-you-own solution would use JavaScript::SpiderMonkey, but that's getting pretty hairy. Another choice is if figuring out how the page processes your userid and password and just crafting your own query string - I'd probably take this approach. Determining success should be as simple as examining the content of the new page.	[reply]
Re^4: What's the easiest way to have perl fill in a web form & submit it? by Anonymous Monk on Feb 09, 2010 at 00:28 UTC
Re^3: What's the easiest way to have perl fill in a web form & submit it? by Corion (Patriarch) on Feb 08, 2010 at 20:41 UTC
Have you looked at the WWW::Mechanize documentation? It shows the ways you can get at the `->content` of the page.	[reply] [d/l]
Re: What's the easiest way to have perl fill in a web form & submit it? by zentara (Cardinal) on Feb 05, 2010 at 13:22 UTC
See Secure Site Login with Mechanize and http digest auth and Using WWW::Mechanize for starters, and google for "WWW::Mechanize tutorial". Its all been done, asked about, and posted before. I'm not really a human, but I play one on earth. Old Perl Programmer Haiku	[reply]