seeing what is going on in form posts

inblosam has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to log in to a site and get a page for some parsing, but I can't seem to mimic everything they are doing in the login (so it doesn't work). I then fell upon this node reply by merlyn (here is original node) and thought I would give it a try. However, I don't quite understand how to use it. I want to see the environment variables when I do a manual login to their site and when I try my own post from an LWP script. This is merlyn's code suggestion:

#!/usr/bin/perl
print "Content-type: text/plain\n\n";
print "$_\t$ENV{$_}\n" for sort keys %ENV;
[download]

Thanks!

Michael Jensen
michael at inshift.com
http://www.inshift.com

Comment on seeing what is going on in form posts Download Code

Replies are listed 'Best First'.
Re: seeing what is going on in form posts by dws (Chancellor) on Jun 26, 2002 at 07:26 UTC
I want to see the environment variables when I do a manual login to their site and when I try my own post from an LWP script. You can't see server-side environment variables unless there's a script on the server side that is prepared to show them to you. If you don't control the server, you're out of luck.	[reply]
(wil) Re: seeing what is going on in form posts by wil (Priest) on Jun 26, 2002 at 09:11 UTC
I want to see the environment variables when I do a manual login to their site and when I try my own post from an LWP script. What exact %ENV variables do you want to know from the remote server? What exactly are you trying to figure out? As others have noted, there is no way to do this using the code or method you referenced in your message. However, there are a few dirty workarounds that might work on some servers to get some or very limited information. LWP::UserAgent could well be of some use to you here. Specifically, you might be interested in a few of it's functions, the most useful might be `$ua->is_protocol_supported($scheme)`. Here's some code snippet that might also be of use in determining the Server: header field returned. Again, this is a dirty hack, and is likely to break or not work at all. `#!/usr/bin/perl use strict; use LWP::UserAgent; my $ua = new LWP::UserAgent; my $req = new HTTP::Request 'HEAD' => 'http://localhost'; $req->header('Accept' => 'text/html'); my $sex_is_good = $ua->request($req); unless (!$sex_is_good->is_success) { print $sex_is_good->as_string; }` [download] Hope this helps! - wil	[reply] [d/l] [select]
Reverse engineering HTTP transactions by Joost (Canon) on Jun 26, 2002 at 11:47 UTC
I am trying to log in to a site and get a page for some parsing, but I can't seem to mimic everything they are doing in the login (so it doesn't work). I've had the same problem a few times, and the easiest to use tool I found to track down the data exchanged is `lynx` - the text-only browser. You can put it in trace-mode and examine all traffic to and from the server. This will probably give you some insight in what's going on (i.e: get/post data, cookies, redirects etc.) See also: `http://lynx.browser.org/ man lynx` [download] `-- Joost downtime n. The period during which a system is error-free and immune from user input.` [download]	[reply] [d/l] [select]
Re: seeing what is going on in form posts by DamnDirtyApe (Curate) on Jun 26, 2002 at 07:27 UTC
I'm not sure you're going to be able to use this code the way you want to. If the code were running on the server you are trying to log on to, then it would be of use. But, unless you are able to put scripts on this remote server, this code is no good to you. AFAIK, there is no way for a browser to unilaterally read the environmental variables of the server. Wouldn't that be a security hole you could drive a bus through? _______________ D a m n D i r t y A p e Home Node \| Email	[reply]
Re: seeing what is going on in form posts by zentara (Cardinal) on Jun 26, 2002 at 16:19 UTC
I'm not sure exactly what you are looking for, the %ENV or the form data. But ........... Mozilla does a real nice job of displaying page info. Click on View -> Page Info -> Forms-> each form It lists all the form's fields and current values.	[reply]