TendulkarIsGod has asked for the wisdom of the Perl Monks concerning the following question:

Please someone help me see the what the h I am doing wrong in this. This might actually end up being a more aspx question than perl, but anyways.

Given command line arguments for a date range, say $to and $from, I am trying to get POST data from a webpage http://somesite.com/foo.aspx. According to the source code for this page, the names of the fields I want to pre-populate are beginDate and endDate

I have a subroutine called build_tree(url) that builds an html::treebuilder tree out of any url passed to it. This is working because I have used it in other scripts. It behaves very well

my code looks like this

$url = "http://www.somesite.com/foo.aspx?beginDate=$from&endDate=$to"; $tree = build_tree($url);

For some reason, this is not behaving like it should. For some reason, the returned content looks like I built a tree from http://somesite.com/foo.aspx, it seems to ignore anything after the '?'.

Can anyone spot what im doing wrong? Is this question more suited to an aspx site? Please help, or please point me in the right direction.

Many thanks

Replies are listed 'Best First'.
Re: POST to an aspx destination
by Corion (Patriarch) on Sep 22, 2010 at 20:37 UTC

    You seem to be constructing a GET request, not a POST request. Does the site return the data you want when you use the URL (as-is) from within your browser?

      Thats exactly the thing, it is not. When I just type in the url as-is into the browser, it's still not getting the content you would expect, which makes me think its really not a Perl question at all.

      I think the reason its not working is the way the form accepts input. Dates can be entered in the fields in 3 different formats into the form. I'm guessing it somehow standardizes this input when you hit the submit button, but to what standard format, I do not know. I have played with a lot of different date formats to try to figure out which one it uses, but other than contacting whoever maintains that site and asking them what format they use, I'm out of ideas.

      Anyways, thanks for the response

        I recommend you learn about how CGI works and how it passes the parameters.

        That said, you can try to use WWW::Mechanize to simulate a browser, or for example WWW::Mechanize::Firefox if Javascript is needed. In both cases, you will still have to understand what goes over the wire and replicate that.

Re: POST to an aspx destination
by zentara (Cardinal) on Sep 23, 2010 at 10:16 UTC
    Hi, here are some notes on how to make a post. These are untested code, by give the general idea.
    > >What's the best way to go about making an HTTP POST? >I want to retrieve a dynamic page that is only accessable via a POST. >Is there a module (simple one?) that deals with this? From 'perldoc lwpcook', use HTTP::Request::Common qw(POST); use LWP::UserAgent; $ua = LWP::UserAgent->new; my $req = POST 'http://www.perl.com/cgi-bin/BugGlimpse', [ search => 'www', errors => 0 ]; print $ua->request($req)->as_string; This is maybe the simplest way I know. Even simpler if there is a web page containing a form that POSTs to th +at page is to use WWW::Mechanize. Then it would be as simple as: use WWW::Mechanize; my $ua = WWW::Mechanize->new; $ua->get( 'http://whatever.example.com/' ); $ua->submit_form( fields => { search => 'www', errors => 0 } ); print $ua->content; (The last line is probably more useful than the literal translation of print $ua->res->as_string; )
    you can also construct a string to post, rather than supplying them as options, as shown in the $relay below
    #!/usr/bin/perl use warnings; use LWP::UserAgent; my $relay; $relay .= "Ecom_transaction_complete=$test&"; $relay .= "IOC_response_code=0&"; my $ua = LWP::UserAgent->new(); my $req = HTTP::Request->new (POST => 'https://zentara.net/cgi-bin/boa +cc.pl'); $req->content_type('application/x-www-form-urlencoded'); $req->content("$relay"); my $res = $ua->request($req); print $res->as_string;

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh