dr_jgbn has asked for the wisdom of the Perl Monks concerning the following question:

Hello,
I am having difficulty with one aspect of an automated form submission. I have managed to successfully submit text into a "textarea" (tested on a simple webform), however I have not been able to figure out how to select for one of the options in a pull down menu which is required to process the form.
For the submission of text I have used this code
my $req = POST 'http://www.ncbi.nlm.nih.gov/Structure/cdd/wrpsb.cgi'; [ SEQUENCE => "$string"];

In the pull down menu there exist 3 parameters to choose from...not sure how to encode my selection for one of these parameters?

Thanks for any help on this issue,
Dr.J

Replies are listed 'Best First'.
Re: Automated form submission
by shotgunefx (Parson) on Mar 02, 2002 at 23:03 UTC
    When processing a POST, the server just gets KEY=VALUE pairs regardless of the form.
    <input type=hidden name="var" value="test">
    <input type=text name="var" value="test">
    <select name="var">
        <option>test</option>
    </select>
    
    
    Submitting any of the above would look the same to the server when it received them. Though if you submitted a value that wasn't on the select list, it might be smart enough to know that. If you know it ahead of time, encode it like any other param.
    my $req = POST $URL, {var=>"value", selectvar=>"selectvalue"};


    -Lee

    "To be civilized is to deny one's nature."
Re: Automated form submission
by Speedy (Monk) on Mar 03, 2002 at 05:06 UTC
    Just put in the value of the variable that you want to change to get a new setting.

    For example, in the NCBI CD-Search URL given above, the database selection variable name and the three possible values can be seen in the "View source" for the HTML page:
    Search Database: <select NAME="DATALIB"> <option VALUE="oasis_smart"> Smart v3.3 - 569 PSSMs <option VALUE="oasis_pfam"> Pfam v6.6 - 3071 PSSMs <option VALUE="oasis_sap" SELECTED> All - 3693 PSSMs </select>
    The variable name is DATALIB, with three possible values: oasis_smart, oasis_pfam, and oasis_sap. The SELECTED beside "oasis_sap" indicates that this is the default or selected value if the user does not change it.

    So in your code, along with the SEQUENCE=>"$string" you also need a DATALIB=>'oasis_smart' if the Smart v3.3 database was to be the new target.

    In fact, you can set all the POST parameters for the wrpsb.cgi page with code like this:
    #!/usr/bin/perl use strict; use HTTP::Request::Common qw(POST); use LWP::UserAgent; my $ua = LWP::UserAgent->new; my $string = ">CG2B_MARGL\n "; $string .= "MLNGENVDSRIMGKVATRASSKGVKSTLGTRGALENISNVARNNLQAGAK\n"; $string .= "KELVKAKRGMTKSKATSSLQSVMGLNVEPMEKAKPQSPEPMDMSEINSAL\n"; $string .= "EAFSQNLLEGVEDIDKNDFDNPQLCSEFVNDIYQYMRKLEREFKVRTDYM\n"; $string .= "TIQEITERMRSILIDWLVQVHLRFHLLQETLFLTIQILDRYLEVQPVSKN\n"; $string .= "KLQLVGVTSMLIAAKYEEMYPPEIGDFVYITDNAYTKAQIRSMECNILRR\n"; $string .= "LDFSLGKPLCIHFLRRNSKAGGVDGQKHTMAKYLMELTLPEYAFVPYDPS\n"; $string .= "EIAAAALCLSSKILEPDMEWGTTLVHYSAYSEDHLMPIVQKMALVLKNAP\n"; $string .= "TAKFQAVRKKYSSAKFMNVSTISALTSSTVMDLADQMC"; my $req=POST 'http://www.ncbi.nlm.nih.gov/Structure/cdd/wrpsb.cgi', [ SEQUENCE=>"$string", DATALIB=>'oasis_sap', INPUT_TYPE=>'fasta', EXPECT=>'0.01', FILTER=>'T', SMODE=>'0', NHITS=>'50', GRAPH=>'2', PAIR=>'2', GW=>'-1']; print "Content-type: text/html\n\n"; print $ua->request($req)->as_string();
    Running this code as, say, sample.cgi in a Web server will return a rather impressive set of match options. Or just running from the command line as, say,
    perl -w sample.cgi
    will give the html source code on the screen.

    Hope this helps. Good luck with this enterprise.
      A little bit off-topic, but your setting of $string is rather laboriuos. Wouldn't it be much easier to use a HERE document, like so:
      my $string=<<EOSTRING; CG2B_MARGL MLNGENVDSRIMGKVATRASSKGVKSTLGTRGALENISNVARNNLQAGAK ... TAKFQAVRKKYSSAKFMNVSTISALTSSTVMDLADQMC EOSTRING
      ?
Re: Automated form submission
by webadept (Pilgrim) on Mar 03, 2002 at 07:14 UTC
    I use this code snipet to send a query request to the monster.com search engine and get the responce. Perhaps you can compare and get your answer from that. Their form is almost all pull down menus
    $request = GET 'http://jobsearch.monster.com/jobsearch.asp', [ ah=>'http://jobsearch.monster.com/', pg=>'1', cq=>$query, col=>'dltci', mx=>'1000', pp=>'500', lid=> '354', cy=>'US', st=>'CA', brd=>'1' ]; $response = $ua->request($request); $r = $response->as_string;


    hope this helps

    webadept.net
Re: Automated form submission
by scain (Curate) on Mar 04, 2002 at 22:18 UTC
    This is not even close to an answer to your question, but it may be easier for you if you get a local copy of CDD and the related web tools. It probably would make the people at NCBI happy to have one less person writing scripts that hammer their website (if your script is going to be used in any sort of high throughput way).

    Scott

      Thanks guys for the response...it seems so obvious now...can't believe I couldn't figure it out
      No worries Scott, I plan to use this program as a model for many other things besides NCBI.
      I am now having another problem that I had working before...I am trying to loop in many individual pieces of text...one at a time, after receiving the response from the server. Here is the code that worked before in a simple manner...
      #!/usr/bin/perl use lib "/System/Library/Perl"; use HTTP::Request::Common qw(POST); use LWP::UserAgent; $ua = LWP::UserAgent->new(); print "Which file do you wish to process?\n"; chomp($file=<STDIN>); open (FILE, "$file") || die "$!"; print "Please enter name for the output file\n"; chomp($out=<STDIN>); open (OUT, ">$out") || die "$!"; while (<FILE>) { @acc = <FILE>; } print "\nPROCESSING file $file...\n"; for ($i=0;$i<=$#acc;$i++) { $accession = $acc[$i]; chomp $accession; etc....I have made the necessary corrections now

      I continue to get an error from the server and I only have one sequence in the input file.
      Any thoughts??
      Thanks again guys!
      Dr.J

        Well, I am not sure of the exact nature of your problem from you description, but I would clean up a few stylistic things, and maybe along the way your problem will clear up too.

        First, when reading in a file you can either do it in a while loop, in which case it will read in line by line, or you can slurp it all at once by assigning the filehandle to an array (each line becoming an element of the array). You've done a hybrid of both here, and that may be the source of your problem. Most people agree that the way to go is to read line by line, that way if you get an input file that is bigger than you expect, you don't pay any penalties for slurping a huge file into memory.

        Also, since you are reading the file in line by line, you can do the rest of the work in the while loop, resulting in:

        print "\nPROCESSING file $file...\n"; while (<FILE>) { chomp $_; $accession = $_; etc....I have made the necessary corrections now }
        If you desided you don't want to do it that way, I would at least change your for loop to a more Perlish loop:
        foreach my $accession(@acc) { chomp $accession; etc....I have made the necessary corrections now }
        Good luck,
        Scott