passing along fields to www::mechanize

coldfingertips has asked for the wisdom of the Perl Monks concerning the following question:

I'm starting a simple script that submits data to some of the lower-end search engines that don't have image codes and all that.

I decided putting the information for each engine in a txt file (see below) would be easiest as I can update information in the future without touching the script itself.

NAME SUBMIT_URL  FORM_ACTION  FORM_NAME  FORM_PARAMS  HIDDEN_PARAMS  S
+UCCESS_TXT
Aesop::http://www.aesop.com/cgi-bin/aesopadd.cgi::GET::1::EMAIL,URL::N
+AME,terms=yes,VERIFIER=072-58::Thank you for using
Amidalla::http://www.amidalla.com/add.htm::POST::1::addurl::::thank yo
+u for
[download]

I just got stumped. I'll be passing a $url, $email, etc to all these engines but each engine calls it something else. So how would I know what variable the URL is supposed to go into if it's not 'url'? It could be V_add, url, URL, pageurl, webaddy, etc.

I can't think of ANY way to do that unless I did so in the txt file.

NAME SUBMIT_URL  FORM_ACTION  FORM_NAME  FORM_PARAMS  HIDDEN_PARAMS  S
+UCCESS_TXT
Aesop::http://www.aesop.com/cgi-bin/aesopadd.cgi::GET::1::EMAIL=$email
+,URL=$url::NAME=$name,terms=yes,VERIFIER=072-58::Thank you for using
Amidalla::http://www.amidalla.com/add.htm::POST::1::addurl=$url::::tha
+nk you for
[download]

Now I think this would work. I could also setup a mysql database, too, which I flirted with for a few minutes. Then I thought that using Mysql to parse all the data (and then splitting some of the fields anyway) would be a bit slower than this text file, and less managable.

One last question. Occassionally there will be one checkbox for an acceptance of terms and the check has a value. Do I have to care what the value is or will www::mechanize click it anyway?

Thanks!

Comment on passing along fields to www::mechanize Select or Download Code

Replies are listed 'Best First'.
Re: passing along fields to www::mechanize by stonecolddevin (Parson) on Feb 28, 2006 at 03:36 UTC
mysql would be MUCH faster than a flat text file. MySQL itself wouldn't be doing the work to parse the data, you'd store it there then extract it with perl. As far as passing variables i'm thinking that each engine is going to take it as a parameter, not a variable name, so the variable is only going to hold the data, not the name. I'm a little stumped as to what exactly you're having trouble with there though... UPDATE: I thought about it a bit, and i've found myself in this situation before. My advice would be to study each of the engine you are submitting to and figure out which values each of them take in their form submissions. "View Source" is your friend :-) meh.	[reply]
Re: passing along fields to www::mechanize by planetscape (Chancellor) on Mar 01, 2006 at 09:03 UTC
I totally agree with dhoss re: MySQL and "View Source"... Also: I would try using a module such as HTTP::Recorder or WWW::Mechanize::Shell to record a successful manual form submission. The output of HTTP::Recorder, for instance, can be "dropped" right into your WWW::Mechanize scripts. Another important tool for finding out what is really happening behind the scenes between server and browser is a protocol analyzer such as Ethereal. HTH, planetscape	[reply]