automation of query with lwp

perl.crazy has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to use Lwp module for automation of website in order to avoid manual submission, but the code is not giving any output. I am not sure where I am doing mistake? The code is as follows:

my $URLtoPostTo="http://www.cbs.dtu.dk/cgi-bin/search.cgi";
my %Fields = (
    "configfile"  =>  "/usr/opt/www/pub/CBS/services/NetMHC-3.0/NetMHC
+.cf",
    "SEQPASTE" =>    ">gi|15607143|Rv0001|ref|NP_214515.1| chromosomal
+ replication initiation protein [Mycobacterium tuberculosis H37Rv]
MTDDPGSGFTTVWNAVVSELNGDPKVDDGPSSDANLSAPLTPQQRAWLNLVQPLTIVEGFALLSVPSSFV
QNEIERHLRAPITDALSRRLGHQIQLGVRIAPPATDEADDTTVPPSENPATTSPDTTTDNDEIDDSAAAR
GDNQHSWPSYFTERPHNTDSATAGVTSLNRRYTFDTFVIGASNRFAHAAALAIAEAPARAYNPLFIWGES
GLGKTHLLHAAGNYAQRLFPGMRVKYVSTEEFTNDFINSLRDDRKVAFKRSYRDVDVLLVDDIQFIEGKE
GIQEEFFHTFNTLHNANKQIVISSDRPPKQLATLEDRLRTRFEWGLITDVQPPELETRIAILRKKAQMER
LAVPDDVLELIASSIERNIRELEGALIRVTAFASLNKTPIDKALAEIVLRDLIADANTMQISAATIMAAT
AEYFDTTVEELRGPGKTRALAQSRQIAMYLCRELTDLSLPKIGQAFGRDHTTVMYAQRKILSEMAERREV
FDHVKELTTRIRQRSKR",
  "peptide" => "on",
   "peplen" => 9,
   "alleles" => "A0202",
  "sort" => "on",
);

use strict;
use LWP::UserAgent;
use HTTP::Request::Common;

my $Browser = new LWP::UserAgent;
my $Page = $Browser->request(POST $URLtoPostTo,\%Fields);

print "Content-type: text/html\n\n";

if ($Page->is_success) 
{ 
      my $content= $Page->content;
      print "$content","\n";
}
[download]

The code is not giving desired output. Please let me know where I am going wroing?

Comment on automation of query with lwp Download Code

Replies are listed 'Best First'.
Re: automation of query with lwp by ccn (Vicar) on Nov 26, 2008 at 19:28 UTC
The problem is not in your code, but rather in parameters provided to 'search.cgi'. Although HTTP::Request::Common want to array reference not hash reference as parameter to 'POST' sub, there is no difference in the server response `my $Page = $Browser->request(POST $URLtoPostTo,[%Fields]);` [download] Make sure that you pass all parameters as webform do. If it doesn't help than take a look on cookies and agent specification	[reply] [d/l]
Re^2: automation of query with lwp by perl.crazy (Initiate) on Nov 26, 2008 at 19:32 UTC
I am novice to this area, tried my best for putting all the parameters. Can you check the webpage and see, I am correct or not? I spent lot of time..all in vain till now. Reading more...will appreciate the help.	[reply]
Re^3: automation of query with lwp by ccn (Vicar) on Nov 26, 2008 at 19:37 UTC
Hmm. Please give an address of that webform.	[reply]
A reply falls below the community's threshold of quality. You may see it by logging in.
A reply falls below the community's threshold of quality. You may see it by logging in.
A reply falls below the community's threshold of quality. You may see it by logging in.
Re: automation of query with lwp by Corion (Patriarch) on Nov 26, 2008 at 19:30 UTC
A more convenient client than LWP::UserAgent would be WWW::Mechanize. It is based on LWP::UserAgent, but it handles things such as Referrer, cookies and form parameters upon submit for you transparently.	[reply]
A reply falls below the community's threshold of quality. You may see it by logging in.