Perl to automate CGI query?

emmiesix has asked for the wisdom of the Perl Monks concerning the following question:

The question is very basic. Can I use perl (about the only 'scripting' language I know) to fill out a CGI form (such as http://fermi.gsfc.nasa.gov/cgi-bin/ssc/LAT/LATDataQuery.cgi) and submit? (Once I get the return page I can handle parsing the html and downloading the data via wget). I see that a cgi perl module exists, but it seems to be geared towards creating your own forms for your own server. I have long been doing big automated downloads using html queries (where the relevant stuff is in the URL), but I'm honestly confused where to go with CGI.

I need to query NASA's server for several thousand objects... a push (or a shove) in the right direction would be appreciated.

Comment on Perl to automate CGI query?

Replies are listed 'Best First'.
Re: Perl to automate CGI query? by ikegami (Patriarch) on Feb 22, 2011 at 22:39 UTC
You can create a request easily using HTTP::Request::Common's `POST`. The resulting object would be passed to LWP::UserAgent. There is also WWW::Mechanize.	[reply] [d/l]
Re^2: Perl to automate CGI query? by emmiesix (Novice) on Jun 24, 2011 at 21:33 UTC
I wanted to come back and say thanks... I tried the LWP method first with no success, but www::mechanize was perfect. I was quite pleased when one of the NASA techs showed a graph with a big spike in it for data downloads... didn't want to tell them it was all due to my script... :)	[reply]
Re: Perl to automate CGI query? by 7stud (Deacon) on Feb 22, 2011 at 23:03 UTC
I just wanted to add: there is no such thing as a "cgi form". There is something called an "html form", and whether it was produced by a cgi script or not is irrelevant.	[reply]
Re^2: Perl to automate CGI query? by Sherm (Sexton) on Feb 22, 2011 at 23:13 UTC
Well, if you want to get really technical about it, there is something called an HTTP POST Request; as is shown by the above answer, whether it was submitted from an HTML form is irrelevant. :-)	[reply]
Re^2: Perl to automate CGI query? by Cody Fendant (Hermit) on Feb 23, 2011 at 04:50 UTC
To be fair, that form submits to a script which has ".cgi" on the end of it, and that's what I think they meant.	[reply]