LWP HTTPS POST returns 500 EOF

epljsod has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: LWP HTTPS POST returns 500 EOF by dominix (Deacon) on Feb 25, 2004 at 01:36 UTC
well ... from a tcp/ip point of view I got this: `telnet login1.swipnet.se 443 Trying 130.244.199.102... telnet: Unable to connect to remote host: Network is unreachable` [download] which elude a lot of trouble-shooting ... but if this server is accessible for you, you may try to add this code `push @{ $ua->requests_redirectable }, 'POST';` [download] this will allow the POST to be redirected, and could save you some more troubleshoot. -- dominix	[reply] [d/l] [select]
Re: Re: LWP HTTPS POST returns 500 EOF by epljsod (Initiate) on Feb 25, 2004 at 22:22 UTC
Hi dominix, first of all, thank you for your response! I tried the requests_redirectable, but unfortunately it did not help. I am able to connect with telnet login1.swipnet.se 443, but I do not know what to write once connected :-) Please enlight me how to perform trouble shooting with telnet. I attach some more debug information that I am able to retreive when using cgi scripts and redirecting the LWP request and the IE request to the dumping cgi scripts. I used normal HTTP, since I can not get HTTPS working with my webserver (not put any effort on it) Do you know any other way to retreive debug information, such as headers? I have in some posts seen more extensive information but I am not able to acquire it. Best regards /Johan From LWP client: BASH=/bin/sh BASH_VERSINFO=([0]="2" [1]="05b" [2]="0" [3]="1" [4]="release" [5]="i3 +86-redhat-linux-gnu") BASH_VERSION='2.05b.0(1)-release' CONTENT_LENGTH=47 CONTENT_TYPE=application/x-www-form-urlencoded DIRSTACK=() DOCUMENT_ROOT=/var/www/html EUID=48 GATEWAY_INTERFACE=CGI/1.1 GROUPS=() HOSTNAME=aca80065.ipt.aol.com HOSTTYPE=i386 HTTP_CONNECTION='TE, close' HTTP_HOST=xxxx.yyyy.org HTTP_TE='deflate,gzip;q=0.3' HTTP_USER_AGENT='Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)' IFS=' ' MACHTYPE=i386-redhat-linux-gnu OPTERR=1 OPTIND=1 OSTYPE=linux-gnu PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/X11R6/bin PIPESTATUS=([0]="0") POSIXLY_CORRECT=y PPID=6272 PS4='+ ' PWD=/var/www/cgi-bin QUERY_STRING= REMOTE_ADDR=213.100.xxx.yyy REMOTE_PORT=63500 REQUEST_METHOD=POST REQUEST_URI=/cgi-bin/print2.cgi SCRIPT_FILENAME=/var/www/cgi-bin/print2.cgi SCRIPT_NAME=/cgi-bin/print2.cgi SERVER_ADDR=172.168.0.101 SERVER_ADMIN=root@localhost SERVER_NAME=xxxx.zzzz.org SERVER_PORT=80 SERVER_PROTOCOL=HTTP/1.1 SERVER_SIGNATURE='<address>Apache/2.0.40 Server at xxxx.yyyy.org Port +80</address> ' SERVER_SOFTWARE='Apache/2.0.40 (Red Hat Linux)' SHELL=/sbin/nologin SHELLOPTS=braceexpand:hashall:interactive-comments:posix SHLVL=1 TERM=dumb UID=48 UNIQUE_ID=mwym9qyoAGUAABiAVDMAAAAB _=echo [download] From IE client: BASH=/bin/sh BASH_VERSINFO=([0]="2" [1]="05b" [2]="0" [3]="1" [4]="release" [5]="i3 +86-redhat-linux-gnu") BASH_VERSION='2.05b.0(1)-release' CONTENT_LENGTH=47 CONTENT_TYPE=application/x-www-form-urlencoded DIRSTACK=() DOCUMENT_ROOT=/var/www/html EUID=48 GATEWAY_INTERFACE=CGI/1.1 GROUPS=() HOSTNAME=aca80065.ipt.aol.com HOSTTYPE=i386 HTTP_ACCEPT='image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, appl +ication/x-shockwave-flash, /' HTTP_ACCEPT_ENCODING='gzip, deflate' HTTP_ACCEPT_LANGUAGE=sv HTTP_CACHE_CONTROL=no-cache HTTP_CONNECTION=Keep-Alive HTTP_HOST=xxxx.yyyy.org HTTP_USER_AGENT='Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)' IFS=' ' MACHTYPE=i386-redhat-linux-gnu OPTERR=1 OPTIND=1 OSTYPE=linux-gnu PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/X11R6/bin PIPESTATUS=([0]="0") POSIXLY_CORRECT=y PPID=27245 PS4='+ ' PWD=/var/www/cgi-bin QUERY_STRING= REMOTE_ADDR=213.100.xxx.yyy REMOTE_PORT=63502 REQUEST_METHOD=POST REQUEST_URI=/cgi-bin/print2.cgi SCRIPT_FILENAME=/var/www/cgi-bin/print2.cgi SCRIPT_NAME=/cgi-bin/print2.cgi SERVER_ADDR=172.168.0.101 SERVER_ADMIN=root@localhost SERVER_NAME=xxxx.yyyy.org SERVER_PORT=80 SERVER_PROTOCOL=HTTP/1.1 SERVER_SIGNATURE='<address>Apache/2.0.40 Server at xxxx.yyyy.org Port +80</address> ' SERVER_SOFTWARE='Apache/2.0.40 (Red Hat Linux)' SHELL=/sbin/nologin SHELLOPTS=braceexpand:hashall:interactive-comments:posix SHLVL=1 TERM=dumb UID=48 UNIQUE_ID=paMUoqyoAGUAAGpttEYAAAAK _=echo [download]	[reply] [d/l] [select]
Re: Re: Re: LWP HTTPS POST returns 500 EOF by dominix (Deacon) on Feb 26, 2004 at 11:51 UTC
the telnet example was to show that there were a connection problem. for me the server is (still) not responding, but anyway, if you want to test some connection over ssl you need no telnet but openssl and type something like : `openssl s_client -connect login1.swipnet.se:443 POST /login.php username=xxxxxxxxxx;password=yyyyyy;` [download] note the CR/LF at the end. I recommend you read more about HTTP if you need to debug such transaction, the next step in debugging is to use a proxy that will do the encryption and you can sniff what happend between your client machine and the proxy , quite tricky anyway. -- dominix	[reply] [d/l]
Re: Re: Re: Re: LWP HTTPS POST returns 500 EOF by epljsod (Initiate) on Feb 26, 2004 at 21:36 UTC
Re: LWP HTTPS POST returns 500 EOF by fizbin (Chaplain) on Feb 25, 2004 at 08:20 UTC
Two suggestions; try them in order and then try them both together: Do a GET request of this same form before you do a POST. It may be that the GET request sets some required cookies. (such as a session ID) In the POST, include also the value for the submit button (Fortsätt). I find this unlikely to be the cause, but you never know in what screwy manner these people wrote their php Actually, the more I think about it, the more I think that (1) is pretty likely the answer. Turning on cookies and then making only one HTTP request is pretty pointless - the other side will see exactly the same traffic from you as if you hadn't enabled cookies. However, if you turn on cookies and then make two requests to the same server, the second request will look different (assuming that the server set a cookie in the first response) If you're still having problems, one other thing I'd check is that internet explorer is able to process this form after having had all cookies from swipnet.se wiped out and having been started directly at the login page (so that it didn't start somewhere else and get a swipnet session cookie that way). If it turns out that internet explorer also needs to, say, visit the top page of http://login1.swipnet.se/ and be redirected in order for the login.php form to work, then I'd just replicate the series of pages that explorer visits with a bunch of GET calls to set the appropriate cookies.	[reply]
Re: Re: LWP HTTPS POST returns 500 EOF by epljsod (Initiate) on Feb 25, 2004 at 22:04 UTC
Hi, thanks for your quick response. Unfortunately, it did not solve my problem. 1. I first tried to add a GET request (see code below), but it did not help. I do not think cookies is the problem, it was just a desperate attempt from my side. I have tried to clear all cookies in IE (were no from swipnet) and set prompt for cookies but it did not try to set any cookies. So I guess the cookie track can be closed. 2. I tried earlier to add the value for the submit button and tried it once more, but it does not seem to be the problem. When using IE I can change the value of the "Fortsätt" button to anything and it still works. Modified code: `use HTTP::Request::Common qw(POST GET); use LWP::UserAgent; use LWP::Debug '+'; my $ua = LWP::UserAgent->new(); $ua->cookie_jar({}); $ua->agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)'); my $req1 = GET 'https://login1.swipnet.se/login.php'; print $ua->request($req1)->as_string; push @{$ua->requests_redirectable }, 'POST'; my $req2 = POST 'https://login1.swipnet.se/login.php', [username => 'xxxxx', password => 'yyyyy', submit => 'Forts']; print $ua->request($req2)->as_string;` [download] Any other suggestions ? Best regards /Johan	[reply] [d/l]
Re: Re: Re: LWP HTTPS POST returns 500 EOF by fizbin (Chaplain) on Feb 26, 2004 at 13:16 UTC
Well, I've looked at your other posts and the only thing I can suggest at this point is to try to match what internet explorer sends absolutely as closely as possible. Unfortunately, you're a little limited by LWP in what you can change about your request - it isn't possible to remove the TE header, nor can you easily force LWP to use HTTP/1.0. (I know, your IE isn't doing that, but IE has been known to use HTTP/1.0 when doing SSL through proxy connections) One thing you can do is add an "Accept" header with a value of "/" to your requests, though if that were really a problem I'd expect your GET to be failing as well. I suppose as one final shot, you could try sending in the request with the login information as a "GET" request. Yes, that's completely different from how IE sends it, but you've tried to be nice and identical, and that's gotten you nowhere. I've got a small https server here on my local network, and I'm going to see what IE and LWP send differently when presented a form like the one you gave in the top post.	[reply]
Re: Re: Re: Re: LWP HTTPS POST returns 500 EOF by epljsod (Initiate) on Feb 26, 2004 at 21:40 UTC
Re: LWP HTTPS POST returns 500 EOF by esskar (Deacon) on Feb 27, 2004 at 21:32 UTC
have you tried setting the REFERER (check back if i spelled it right) field in the HTTP Request Header?	[reply]
Re: Re: LWP HTTPS POST returns 500 EOF by epljsod (Initiate) on Feb 29, 2004 at 20:16 UTC
Hi all! The problem has finally been solved. I would like to thank every one for your effort and contributions. I could not have solved the problem without your help! After the laborations with curl, I felt that the problem must be connected to the ciphering. I knew that the only way it worked with curl was when I set the -2 option forcing it to use SSL ver 2. Therefore I today read some more about the cihpering in LWP and tried to set SSL ver 2 also in LWP ($ENV{HTTPS_VERSION} = '2') and tada, it finally worked :-) Actually, I already tried this option yesterday, but then I ran through the recently installed proxy and that probably screwed thing up somehow. Hope this thread can help others with same problem. I have certainly learned a lot while struggling with this. Best regards /Johan	[reply]
Re^3: LWP HTTPS POST returns 500 EOF by FrankRizzo (Acolyte) on Aug 19, 2005 at 12:36 UTC
I'm having this exact problem, and none of the suggestions in this thread have helped. I'm also having a fight with curl. Would someone who has curl working properly give this a shot? curl -d '__VIEWSTATE=dDw1MzgxO3Q8O2w8aTwwPjs\x2BO2w8dDw7bDxpPDEzPjs\x2 +BO2w8dDw7bDxpPDk\x2BOz47bDx0PDtsPGk8OT47PjtsPHQ8O2w8aTw3Pjs\x2BO2w8dD +w7bDxpPDA\x2BOz47bDx0PDtsPGk8MD47PjtsPHQ8O2w8aTwxPjs\x2BO2w8dDw7bDxpP +DI\x2BO2k8Mz47aTw0PjtpPDU\x2BO2k8Nj47aTw3PjtpPDg\x2BO2k8OT47aTwxMD47P +jtsPHQ8cDxwPGw8X2N0bDVfX2N0bDNfX2N0bDBfRm9ybUl0ZW1fMjA1MTA3NF9SZXF1aX +JlZEZpZWxkVmFsaWRhdG9yX19jdGw1X19jdGwzX19jdGwwX0Zvcm1JdGVtXzIwNTEwNzR +fTGFiZWxfRm9yZUNvbG9yO0Rpc3BsYXk7PjtsPDI8XGU\x2BO1N5c3RlbS5XZWIuVUkuV +2ViQ29udHJvbHMuVmFsaWRhdG9yRGlzcGxheSwgU3lzdGVtLldlYiwgVmVyc2lvbj0xLj +AuNTAwMC4wLCBDdWx0dXJlPW5ldXRyYWwsIFB1YmxpY0tleVRva2VuPWIwM2Y1ZjdmMTF +kNTBhM2E8Tm9uZT47Pj47Pjs7Pjt0PHA8cDxsPF9jdGw1X19jdGwzX19jdGwwX0Zvcm1J +dGVtXzIwNTEwNzVfUmVxdWlyZWRGaWVsZFZhbGlkYXRvcl9fY3RsNV9fY3RsM19fY3RsM +F9Gb3JtSXRlbV8yMDUxMDc1X0xhYmVsX0ZvcmVDb2xvcjtEaXNwbGF5Oz47bDwyPFxlPj +s1MDxOb25lPjs\x2BPjs\x2BOzs\x2BO3Q8cDxwPGw8X2N0bDVfX2N0bDNfX2N0bDBfRm +9ybUl0ZW1fMjA1MTA3Nl9SZXF1aXJlZEZpZWxkVmFsaWRhdG9yX19jdGw1X19jdGwzX19 +jdGwwX0Zvcm1JdGVtXzIwNTEwNzZfTGFiZWxfRm9yZUNvbG9yO0Rpc3BsYXk7PjtsPDI8 +XGU\x2BOzUwPE5vbmU\x2BOz4\x2BOz47Oz47dDxwPHA8bDxfY3RsNV9fY3RsM19fY3Rs +MF9Gb3JtSXRlbV8yMDUxMDc3X1JlcXVpcmVkRmllbGRWYWxpZGF0b3JfX2N0bDVfX2N0b +DNfX2N0bDBfRm9ybUl0ZW1fMjA1MTA3N19MYWJlbF9Gb3JlQ29sb3I7RGlzcGxheTs\x2 +BO2w8MjxcZT47NTA8Tm9uZT47Pj47Pjs7Pjt0PHA8cDxsPF9jdGw1X19jdGwzX19jdGww +X0Zvcm1JdGVtXzIwNTEwNzhfUmVxdWlyZWRGaWVsZFZhbGlkYXRvcl9fY3RsNV9fY3RsM +19fY3RsMF9Gb3JtSXRlbV8yMDUxMDc4X0xhYmVsX0ZvcmVDb2xvcjtEaXNwbGF5Oz47bD +wyPFxlPjs1MDxOb25lPjs\x2BPjs\x2BOzs\x2BO3Q8cDxwPGw8X2N0bDVfX2N0bDNfX2 +N0bDBfRm9ybUl0ZW1fMjA1MTA3OV9SZWd1bGFyRXhwcmVzc2lvblZhbGlkYXRvcl9fY3R +sNV9fY3RsM19fY3RsMF9Gb3JtSXRlbV8yMDUxMDc5X0xhYmVsX0ZvcmVDb2xvcjtEaXNw +bGF5Oz47bDwyPFxlPjs1MDxOb25lPjs\x2BPjs\x2BOzs\x2BO3Q8cDxwPGw8X2N0bDVf +X2N0bDNfX2N0bDBfRm9ybUl0ZW1fMjA1MTA3OV9SZXF1aXJlZEZpZWxkVmFsaWRhdG9yX +19jdGw1X19jdGwzX19jdGwwX0Zvcm1JdGVtXzIwNTEwNzlfTGFiZWxfRm9yZUNvbG9yO0 +Rpc3BsYXk7PjtsPDI8XGU\x2BOzUwPE5vbmU\x2BOz4\x2BOz47Oz47dDxwPHA8bDxfY3 +RsNV9fY3RsM19fY3RsMF9Gb3JtSXRlbV8yMDUxMDgwX1JlZ3VsYXJFeHByZXNzaW9uVmF +saWRhdG9yX19jdGw1X19jdGwzX19jdGwwX0Zvcm1JdGVtXzIwNTEwODBfTGFiZWxfRm9y +ZUNvbG9yO0Rpc3BsYXk7PjtsPDI8XGU\x2BOzUwPE5vbmU\x2BOz4\x2BOz47Oz47dDxw +PHA8bDxfY3RsNV9fY3RsM19fY3RsMF9Gb3JtSXRlbV8yMDUxMDgxX1JlZ3VsYXJFeHByZ +XNzaW9uVmFsaWRhdG9yX19jdGw1X19jdGwzX19jdGwwX0Zvcm1JdGVtXzIwNTEwODFfTG +FiZWxfRm9yZUNvbG9yO0Rpc3BsYXk7PjtsPDI8XGU\x2BOzUwPE5vbmU\x2BOz4\x2BOz +47Oz47Pj47Pj47Pj47Pj47Pj47Pj47Pj47Pj47Pj47PndIUNKupgylSQgKnJm59RrmMvd +T' -d '_ctl5\x3A_ctl3\x3A_ctl0\x3AFormItem_2051074=First' -d '_ctl5\x +3A_ctl3\x3A_ctl0\x3AFormItem_2051075=Last' -d '_ctl5\x3A_ctl3\x3A_ctl +0\x3AFormItem_2051076=Address' -d '_ctl5\x3A_ctl3\x3A_ctl0\x3AFormIte +m_2051077=City' -d '_ctl5\x3A_ctl3\x3A_ctl0\x3AFormItem_2051078=State +' -d '_ctl5\x3A_ctl3\x3A_ctl0\x3AFormItem_2051079=90210' -d '_ctl5\x3 +A_ctl3\x3A_ctl0\x3AFormItem_2051080=joe@netcom.com' -d '_ctl5\x3A_ctl +3\x3A_ctl0\x3AFormItem_2051081=213-312-2132' -d '_ctl5\x3A_ctl3\x3A_c +tl0\x3AFormItem_2051082=Color' -d '_ctl5\x3A_ctl3\x3A_ctl0\x3AFormIte +m_2051083=Animal' -d '_ctl5\x3A_ctl3\x3A_ctl0\x3AFormItem_2051084=E-m +ail' -d '_ctl5\x3A_ctl3\x3A_ctl0\x3AFormItem_2051085="New Car"' -d '_ +ctl5\x3A_ctl3\x3A_ctl0\x3AFormItem_2051086="MINI Cooper"' -d '_ctl5\x +3A_ctl3\x3A_ctl0\x3AFormItem_2051087=Message' -d '_ctl5\x3A_ctl3\x3A_ +ctl0\x3AForm_121155_Submit=Submit' --trace out.txt https://secure.web +makerx.net/Contact-Us.aspx?hostname=billjacobsmini.com [download] When I run it, I get "curl: (35) SSL: couldn't set callback!". I saved off the html from the website in question, and modified the post method to post to my local webserver, which was running the CGI app from the LWP book, which showed all the stuff being POSTed. Yes, they actually USE that crappy long string, and it's always the same. When I try to POST to the real site, I get "500 Internal Server Error". ANY help is appreciated!	[reply] [d/l]
Re: LWP HTTPS POST returns 500 EOF by dominix (Deacon) on Feb 27, 2004 at 12:04 UTC
sounds weird, did you try with curl ? just to know ? -- dominix	[reply]
Re: Re: LWP HTTPS POST returns 500 EOF by epljsod (Initiate) on Feb 27, 2004 at 21:14 UTC
Hi again dominix! Finally, I think we have a little breakthrough. I laborated a bit with curl (never heard of it before) and when I issued the following command line it was able to correctly log in :) `[johan@aca80065 ~]$ curl -d 'username=xxxx' -d 'password=yyyyy' -2 htt +ps://login1.swipnet.se/login.php` [download] However, I am still very interested to know why it does not work with LWP, since I think I still prefer to use LWP. Does this information give you any clues of what the problem might be using LWP ? Here is a log of the sent data (headers) with curl [root@aca80065 johan]# less log.txt == Info: About to connect() to login1.swipnet.se:443 == Info: Connected to login1.swipnet.se (130.244.199.102) port 443 == Info: SSL connection using DES-CBC3-MD5 == Info: Server certificate: == Info: subject: /C=SE/ST=Stockholm/L=Kista/O=Tele2 Sverige A +B/CN=login1.swipnet.se/emailA ddress=staff@swip.net == Info: start date: 2002-03-14 15:23:52 GMT == Info: expire date: 2004-03-13 15:23:52 GMT == Info: common name: login1.swipnet.se (matched) == Info: issuer: /C=ZA/ST=Western Cape/L=Cape Town/O=Thawte Co +nsulting cc/OU=Certification Services Division/CN=Thawte Server CA/emailAddress=server-certs@thawte +.com => Send header, 334 bytes (0x14e) 0000: 50 4f 53 54 20 2f 6c 6f 67 69 6e 2e 70 68 70 20 POST /login.php 0010: 48 54 54 50 2f 31 2e 31 0d 0a 55 73 65 72 2d 41 HTTP/1.1..User-A 0020: 67 65 6e 74 3a 20 63 75 72 6c 2f 37 2e 39 2e 38 gent: curl/7.9.8 0030: 20 28 69 33 38 36 2d 72 65 64 68 61 74 2d 6c 69 (i386-redhat-li 0040: 6e 75 78 2d 67 6e 75 29 20 6c 69 62 63 75 72 6c nux-gnu) libcurl 0050: 20 37 2e 39 2e 38 20 28 4f 70 65 6e 53 53 4c 20 7.9.8 (OpenSSL 0060: 30 2e 39 2e 37 61 29 20 28 69 70 76 36 20 65 6e 0.9.7a) (ipv6 en 0070: 61 62 6c 65 64 29 0d 0a 48 6f 73 74 3a 20 6c 6f abled)..Host: lo 0080: 67 69 6e 31 2e 73 77 69 70 6e 65 74 2e 73 65 0d gin1.swipnet.se. 0090: 0a 50 72 61 67 6d 61 3a 20 6e 6f 2d 63 61 63 68 .Pragma: no-cach 00a0: 65 0d 0a 41 63 63 65 70 74 3a 20 69 6d 61 67 65 e..Accept: image 00b0: 2f 67 69 66 2c 20 69 6d 61 67 65 2f 78 2d 78 62 /gif, image/x-xb 00c0: 69 74 6d 61 70 2c 20 69 6d 61 67 65 2f 6a 70 65 itmap, image/jpe 00d0: 67 2c 20 69 6d 61 67 65 2f 70 6a 70 65 67 2c 20 g, image/pjpeg, 00e0: 2a 2f 2a 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e /..Content-Len 00f0: 67 74 68 3a 20 33 34 0d 0a 43 6f 6e 74 65 6e 74 gth: 34..Content 0100: 2d 54 79 70 65 3a 20 61 70 70 6c 69 63 61 74 69 -Type: applicati 0110: 6f 6e 2f 78 2d 77 77 77 2d 66 6f 72 6d 2d 75 72 on/x-www-form-ur 0120: 6c 65 6e 63 6f 64 65 64 0d 0a 0d 0a 75 73 65 72 lencoded....user 0130: 6e 61 6d 65 3d 6b 2d 35 31 38 37 36 26 70 61 73 name=xxxx&pas 0140: 73 77 6f 72 64 3d 35 65 52 2d 6f 52 2d 75 sword=yyyyy <= Recv header, 17 bytes (0x11) 0000: 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK. 0010: 0a . <= Recv header, 37 bytes (0x25) 0000: 44 61 74 65 3a 20 46 72 69 2c 20 32 37 20 46 65 Date: Fri, 27 Fe 0010: 62 20 32 30 30 34 20 32 30 3a 35 38 3a 34 34 20 b 2004 20:58:44 0020: 47 4d 54 0d 0a GMT.. <= Recv header, 70 bytes (0x46) 0000: 53 65 72 76 65 72 3a 20 41 70 61 63 68 65 2f 31 Server: Apache/1 0010: 2e 33 2e 32 38 20 28 55 6e 69 78 29 20 50 48 50 .3.28 (Unix) PHP 0020: 2f 34 2e 33 2e 32 20 6d 6f 64 5f 73 73 6c 2f 32 /4.3.2 mod_ssl/2 0030: 2e 38 2e 31 35 20 4f 70 65 6e 53 53 4c 2f 30 2e .8.15 OpenSSL/0. 0040: 39 2e 36 69 0d 0a 9.6i.. <= Recv header, 25 bytes (0x19) 0000: 58 2d 50 6f 77 65 72 65 64 2d 42 79 3a 20 50 48 X-Powered-By: PH 0010: 50 2f 34 2e 33 2e 32 0d 0a P/4.3.2.. <= Recv header, 28 bytes (0x1c) 0000: 54 72 61 6e 73 66 65 72 2d 45 6e 63 6f 64 69 6e Transfer-Encodin 0010: 67 3a 20 63 68 75 6e 6b 65 64 0d 0a g: chunked.. <= Recv header, 25 bytes (0x19) 0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te 0010: 78 74 2f 68 74 6d 6c 0d 0a xt/html.. [download] My suspects are Accept (which I recognize values from IE), no cache, ciphering type (longshot), URL base. Is the only way to investigate the sent HTTP headers with LWP to install a proxy ? It would be great with a -trace flag as in curl. Best regards /Johan	[reply] [d/l] [select]
Re: Re: Re: LWP HTTPS POST returns 500 EOF by epljsod (Initiate) on Feb 27, 2004 at 22:51 UTC
I installed a Proxy (Proxy Workbench) and it seems to be some major problems with the communication. When I connected with curl, the connection started with a CONNECT message with the following content: 000000 CONNECT 067 079 078 078 069 067 084 032 000008 login1.s 108 111 103 105 110 049 046 115 000016 wipnet.s 119 105 112 110 101 116 046 115 000024 e:443 HT 101 058 052 052 051 032 072 084 000032 TP/1.0.. 084 080 047 049 046 048 013 010 000040 User-Age 085 115 101 114 045 065 103 101 000048 nt: curl 110 116 058 032 099 117 114 108 000056 /7.9.8 ( 047 055 046 057 046 056 032 040 000064 i386-red 105 051 056 054 045 114 101 100 000072 hat-linu 104 097 116 045 108 105 110 117 000080 x-gnu) l 120 045 103 110 117 041 032 108 000088 ibcurl 7 105 098 099 117 114 108 032 055 000096 .9.8 (Op 046 057 046 056 032 040 079 112 000104 enSSL 0. 101 110 083 083 076 032 048 046 000112 9.7a) (i 057 046 055 097 041 032 040 105 000120 pv6 enab 112 118 054 032 101 110 097 098 000128 led).... 108 101 100 041 013 010 013 010 [download] While the LWP connection attempt started with a POST message: 000000 POST htt 080 079 083 084 032 104 116 116 000008 ps://log 112 115 058 047 047 108 111 103 000016 in1.swip 105 110 049 046 115 119 105 112 000024 net.se/l 110 101 116 046 115 101 047 108 000032 ogin.php 111 103 105 110 046 112 104 112 000040 HTTP/1. 032 072 084 084 080 047 049 046 000048 1..TE: d 049 013 010 084 069 058 032 100 000056 eflate,g 101 102 108 097 116 101 044 103 000064 zip;q=0. 122 105 112 059 113 061 048 046 000072 3..Conne 051 013 010 067 111 110 110 101 t 000080 ction: T 099 116 105 111 110 058 032 084 000088 E, close 069 044 032 099 108 111 115 101 000096 ..Host: 013 010 072 111 115 116 058 032 000104 login1.s 108 111 103 105 110 049 046 115 000112 wipnet.s 119 105 112 110 101 116 046 115 000120 e..User- 101 013 010 085 115 101 114 045 000128 Agent: M 065 103 101 110 116 058 032 077 000136 ozilla/4 111 122 105 108 108 097 047 052 000144 .0 (comp 046 048 032 040 099 111 109 112 000152 atible; 097 116 105 098 108 101 059 032 000160 MSIE 6.0 077 083 073 069 032 054 046 048 000168 ; Window 059 032 087 105 110 100 111 119 000176 s NT 5.1 115 032 078 084 032 053 046 049 000184 )..Conte 041 013 010 067 111 110 116 101 000192 nt-Lengt 110 116 045 076 101 110 103 116 000200 h: 47..C 104 058 032 052 055 013 010 067 000208 ontent-T 111 110 116 101 110 116 045 084 000216 ype: app 121 112 101 058 032 097 112 112 000224 lication 108 105 099 097 116 105 111 110 000232 /x-www-f 047 120 045 119 119 119 045 102 000240 orm-urle 111 114 109 045 117 114 108 101 000248 ncoded.. 110 099 111 100 101 100 013 010 000256 .. 013 010 [download] The content transmitted for the working curl connection was unreadable after the CONNECT message which gave me the impression that the transmission was ciphered. In other words, it seems to me that LWP does not use ciphering (and perhaps also incorrect protocol). I have searched the man pages for LWP::UserAgent, but can not find anything relevant. Does this help ? Found the following in man Crypt::SSLeay LWP::UserAgent has its own methods of proxying which may work for you +and is likely incompatible with Crypt::SSLeay proxy support. To use LWP::UserAgent proxy support, try something like: my $ua = new LWP::UserAgent; $ua->proxy([qw( https http )], "$proxy_ip:$proxy_port"); At the time of this writing, libwww v5.6 seems to proxy https requests + fine with an Apache mod_proxy server. It sends a line like: GET https://www.nodeworks.com HTTP/1.1 to the proxy server, which is not the CONNECT request that some proxie +s would expect, so this may not work with other proxy servers than mo +d_proxy. The CONNECT method is used by Crypt::SSLeayâs internal prox +y support. [download] Which makes me even more confused :-/ Best regards /Johan	[reply] [d/l] [select]