how to decide the URL

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I want to use "post" to submit my sequence automatically to http://www.ncbi.nlm.nih.gov/Structure/cdd/wrpsb.cgi. What kind of URL will I use? what kind of content should I use? $req->content('match=www&errors=0'); Please Help! The code is:

#!/usr/bin/perl

use LWP::Simple;
use LWP::UserAgent;

$ua = new LWP::UserAgent;
my $req = new HTTP::Request 'POST','http://www.ncbi.nlm.nih.gov/htbin-
+post/Entrez/query?uid=9254694&form=6&db=m&Dopt=r';
$req->content_type('application/x-www-form-urlencoded');
$req->content('match=www&errors=0');

my $res = $ua->request($req);
print $res->as_string,"\n";
[download]

Here is the output:

HTTP/1.1 200 OK
Connection: close
Date: Tue, 08 Apr 2003 02:33:07 GMT
Via: 1.1 www.ncbi.nih.gov
Server: Apache
Content-Type: text/html
Client-Date: Tue, 08 Apr 2003 02:33:28 GMT
Client-Peer: 130.14.29.110:80
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
Title: Entrez Error
X-Cache: MISS from www.ncbi.nih.gov

<html>
<head>
<TITLE>Entrez Error</TITLE>
</head>
<body>
<HR>
<H1>Error</H1><P>
This link contained an data type unknown to Entrez.
</body>
</html>
[download]

Thanks in advance!

20030408 Edit by Corion: Added second pair of code tags

Comment on how to decide the URL Select or Download Code

Replies are listed 'Best First'.
Re: how to decide the URL by Corion (Patriarch) on Apr 08, 2003 at 06:55 UTC
Depending on how your "target" is implemented, it is a bad idea to mix GET and POST parameters. Personally, I wouldn't use LWP::UserAgent anymore for web automation but WWW::Mechanize, which shields you from those mundane tasks of figuring out the values and names (No, personally, I use my own module, WWW::Mechanize::Shell, but that's just a frontend to WWW::Mechanize). I tried a plain GET url in my browser and that dosen't work, so for further diagnosis, please also post the original url (the one giving you the form you're trying to fill out). `perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web` [download]	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: how to decide the URL
by Corion (Patriarch) on Apr 08, 2003 at 06:55 UTC

Depending on how your "target" is implemented, it is a bad idea to mix GET and POST parameters. Personally, I wouldn't use LWP::UserAgent anymore for web automation but WWW::Mechanize, which shields you from those mundane tasks of figuring out the values and names (No, personally, I use my own module, WWW::Mechanize::Shell, but that's just a frontend to WWW::Mechanize).

I tried a plain GET url in my browser and that dosen't work, so for further diagnosis, please also post the original url (the one giving you the form you're trying to fill out).

perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web
[download]

[reply]
[d/l]
[select]