in reply to Re: submit Form with Lynx but not Perl?
in thread submit Form with Lynx but not Perl?

Thanks for th reply, but allowing redirection does not solve the problem. While Lynx get the next page - the Perl-program gets again the initial page.
Could it be that Lynx writes a coockie on the server, while Perl doesn't do that? Lynx is set that it does not accept any cookie - so no cookie is written on my computer.

This is from Lynx:

LYCookie: Searching for 'subscriber.chello.at:80', '/sms/index.php3'. Composing Authorization for subscriber.chello.at:80/sms/index.php3 HTAASetup_lookup: No template matched `sms/index.php3' (so probably no +t protected) HTTP: Not sending authorization (yet). Writing: GET /sms/index.php3 HTTP/1.0 Host: subscriber.chello.at .... ---------------------------------- HTTP Request is sent. HTTP: WRITE delivered OK HTTP Request sent; waiting for answer. HTTP: Trying to read 1535 HTTP: Read 1448 HTTP: Rx: HTTP/1.1 200 OK HTTP: Scanned 2 fields from line_buffer --- Talking HTTP1. HTTP/1.1 200 OK .... Connection: close
this I got using Perl (I set in Perl the User-Agent to Lynx 2.8.4rel.1, it could be ...):
_protocol => HTTP/1.1 _request => HTTP::Request=HASH(0x84e226c) _headers => HTTP::Headers=HASH(0x84e8fac) _previous => HTTP::Response=HASH(0x84dfda0) _msg => OK _rc => 200
The content of the response (rc=200) is the initial web-page. It is not the next form to enter the sms text.

Here are the Information of Previous, Headers and Request:

Previous: _protocol => HTTP/1.1 _request => HTTP::Request=HASH(0x84e22a8) _headers => HTTP::Headers=HASH(0x84dfd94) _rc => 302 _msg => Found _content => Previous-Headers: client-transfer-encoding => chunked client-response-num => 1 location => /sms/index.php3 content-type => text/html server => Apache/1.3.26 (Unix) PHP/4.2.1 connection => close date => Tue, 06 Aug 2002 09:28:00 GMT x-powered-by => PHP/4.2.1 client-date => Tue, 06 Aug 2002 09:28:00 GMT Previous-Request: _method => POST _headers => HTTP::Headers=HASH(0x84e229c) _uri => http://subscriber.chello.at/sms/austria.php3 _content => austria=43699&image.x=0&image.y=0 Previous-Request-Headers: user-agent => Lynx/2.8.4rel.1 content-type => application/x-www-form-urlencoded content-length => 33 - - - - - - Request: _method => POST _headers => HTTP::Headers=HASH(0x84dfaac) _uri => http://subscriber.chello.at/sms/index.php3 _content => austria=43699&image.x=0&image.y=0 Request-Headers: user-agent => Lynx/2.8.4rel.1 content-type => application/x-www-form-urlencoded content-length => 33 - - - - - - Headers: client-transfer-encoding => chunked client-response-num => 1 title => hello, i'm chello content-type => text/html server => Apache/1.3.26 (Unix) PHP/4.2.1 x-meta-author => chello broadband B.V 1999 date => Tue, 06 Aug 2002 09:28:05 GMT connection => close x-powered-by => PHP/4.2.1 link => </dhtml/style/sliding.css>; rel="stylesheet"; type="text/css" client-date => Tue, 06 Aug 2002 09:28:10 GMT

Replies are listed 'Best First'.
Re: Re: Re: submit Form with Lynx but not Perl?
by grantm (Parson) on Aug 06, 2002 at 21:15 UTC

    No, it's not a cookie problem - I can't see any cookie headers in your trace output (and as you say, you have cookies disabled in Lynx anyway).

    Since you're being redirected back to the original page, it looks like there is something wrong with your initial request. My approach would be to use a tool like tcpdump on Linux or Analyzer on Windows to trace the network and compare the Lynx request to the LWP request.

      Hi,

      Analysing the traffic difference between Perl and Lynx gives me this:

      Lynx sends in one frame POST & URL & field values.
      while Perl sends only POST & URL without field values,
      then Perl waits(?) for an 'ACK'
      and then it sends the field values in another frame and is redirected to the initial page.

      Is there anything to change this behviour?
      This is my code to to get the next form:

      BEGIN { *LWP::UserAgent::redirect_ok = sub {1} } $ub = LWP::UserAgent->new; $ub->agent($MOZILLA); $res = $ub->request($form[1]->click('image',0,0)); # or this: $res = $ub->request(POST $URL, @TEL);
      Again thank you very much for the support,
      it helped me a lot,
      Carl

      LYNX-TRAFFIC:

      Frame 7 (668 on wire, 668 captured) .. Hypertext Transfer Protocol POST /sms/austria.php3 HTTP/1.0\r\n Host: subscriber.chello.at\r\n Accept: text/html, text/plain, application/x-perl, image/jpeg, ima +ge/*, application/x-gunzip, application/x-gzip, application/x-bunzip2 +, application/x-tar-gz, video/*, text/sgml, application/postscript, * +/*;q=0.01\r\n Accept-Encoding: gzip, compress\r\n Accept-Language: en\r\n Pragma: no-cache\r\n Cache-Control: no-cache\r\n User-Agent: Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0. +9.6c\r\n Referer: http://subscriber.chello.at/sms/index.php3\r\n Content-type: application/x-www-form-urlencoded\r\n Content-length: 33\r\n \r\n Data (33 bytes) 0000 61 75 73 74 72 69 61 3d 34 33 36 39 39 26 69 6d austria=43699& +im 0010 61 67 65 2e 78 3d 30 26 69 6d 61 67 65 2e 79 3d age.x=0&image. +y= 0020 30 0
      PERL-TRAFFIC:
      Frame 7 (246 on wire, 246 captured) ... Hypertext Transfer Protocol POST /sms/austria.php3 HTTP/1.1\r\n Connection: close\r\n Host: subscriber.chello.at\r\n User-Agent: Perl/sms.pl\r\n Content-Length: 33\r\n Content-Type: application/x-www-form-urlencoded\r\n \r\n Frame 8 (66 on wire, 66 captured) .. Flags: 0x0010 (ACK) 0... .... = Congestion Window Reduced (CWR): Not set .0.. .... = ECN-Echo: Not set ..0. .... = Urgent: Not set ...1 .... = Acknowledgment: Set .... 0... = Push: Not set .... .0.. = Reset: Not set .... ..0. = Syn: Not set .... ...0 = Fin: Not set .. Frame 9 (99 on wire, 99 captured) .. Hypertext Transfer Protocol Data (33 bytes) 0000 61 75 73 74 72 69 61 3d 34 33 36 39 39 26 69 6d austria=4369 +9&im 0010 61 67 65 2e 78 3d 30 26 69 6d 61 67 65 2e 79 3d age.x=0&imag +e.y= 0020 30 0 Frame 10 (294 on wire, 294 captured) .. Hypertext Transfer Protocol HTTP/1.1 302 Found\r\n Date: Wed, 07 Aug 2002 10:15:45 GMT\r\n Server: Apache/1.3.26 (Unix) PHP/4.2.1\r\n X-Powered-By: PHP/4.2.1\r\n Location: /sms/index.php3\r\n Connection: close\r\n Transfer-Encoding: chunked\r\n Content-Type: text/html\r\n \r\n Data (5 bytes) 0000 30 0d 0a 0d 0a 0....

        I don't think the fact that the TCP stream is fragmented into multiple packets is likely to be causing your problem.

        One possibility is that the server logic expects a referer header:

        Referer: http://subscriber.chello.at/sms/index.php3
        I've got it!

        The servers denied to go on unless the correct Referer is entered in the header. LWP does not do by it self.

        Thanks for your hints,
        Carl

      Thanks,
      I'll try tcpdump - that means lol (=lot of lines).

      I'll let you know, if you are interested,
      Carl
      (carl.schreiber@web.de)