in reply to LWP::UserAgent destroys $response->content >:{

Looking at the log more carefully I see that it isn't, in fact, a terminal, superfluous request from the UserAgent client. It looks more like a server error being reported in response to the UseAgent request.

So, what, then, is UserAgent doing to mangle the request that provokes the error?

After all, in filename method I DO get the full expected response in the rspfile (but not in the $response->content).

Do I need to add some (as yet undetermined, and possible blank/dummy) "entity body" to the request?

Steve Potter
http://www3.sympatico.ca/steven.potter/
  • Comment on Re: LWP::UserAgent destroys $response->content >:{

Replies are listed 'Best First'.
Re: Re: LWP::UserAgent destroys $response->content >:{
by iguanodon (Priest) on May 22, 2003 at 14:23 UTC
    I don't think this is an LWP::UserAgent problem. It may be something specific to the Linksys "web server". I can grab the status page from my Netgear router with no problem using this code:
    #!/usr/local/bin/perl use strict; use warnings; $| = 1; use LWP::UserAgent; use HTTP::Request::Common qw(GET POST); use HTTP::Cookies; my $user = 'user'; my $pass = 'pass'; my $cookie_jar = HTTP::Cookies->new(file => '/tmp/cookie_jar', AutoSav +e => 1); my $ua = LWP::UserAgent->new(); $ua->cookie_jar($cookie_jar); my $req = GET 'http://192.168.0.1/mtenSysStatistics.html'; $req->authorization_basic($user, $pass); my $response = $ua->request($req); print $response->code . ' ' . $response->message . "\n"; my $content = $response->content; if ($content) { print $content; } else { print 'No content!'; } print "\n";

    Can you give this a try (after adjusting for your environment) and see if you still get the 500 error?

      I added the cookie jar (relavant code below)
      (I didn't spot any other functional difference)

      Same Result :{

      So, it looks like I'll have to dispense with the
      $response = $ua->request($request);
      and replace it with low level code - merde! ;)

      use AppConfig; use File::Basename; use Getopt::Long; use HTML::Form; use HTTP::Request::Common; use HTML::TreeBuilder; use HTTP::Cookies; use LWP::DebugFile ('+'); use LWP::UserAgent; use Pod::Usage; use URI::file; . . my $cookie_jar = HTTP::Cookies->new(file => '/tmp/cookie_jar', AutoSav +e => 1); my $ua; . . $ua = LWP::UserAgent->new(); $ua->agent('Mozilla/4.0'); $ua->max_size('5000'); $ua->cookie_jar($cookie_jar); . . $ua->credentials($netloc, $realm, $userid, $passwd); . . $request = HTTP::Request->new(GET => "http://192.168.1.1/Status.htm"); $response = $ua->request($request); #$response = $ua->request($request,"rspfile.html"); if ($response->is_error) { my $string; print "Error: " . $response->status_line . "\n"; print $response->error_as_HTML; $string = $response->as_string; die "$string\n"; } . .

      Steve Potter
      http://www3.sympatico.ca/steven.potter/
Re: Re: LWP::UserAgent destroys $response->content >:{
by Thelonius (Priest) on May 22, 2003 at 14:42 UTC
    No, it is LWP that is giving the error "Can't read entity body". That is from LWP/Protocol/http.pm (line 338 in the version I have). It is giving that error because the LinkSys reset the socket connection instead of closing it (or at least LWP thinks that's what happened). It could be a bug in LWP or the FreeBSD TCP/IP layer, but it could also be a bug in the LinkSys (somewhat more likely, I think).
      So, Shall I dispense with the UserAgent Request and run it through lower level code? >:-{ (Defeating the purpose of UserAgent) (Not a satisfactory solution)

      OR

      Use the filename method and use a 'length of file (= length of response)' test before assuming that UserAgent error response is valid (i.e. rebuild the response->content from the collected file copy if response is not 'too short'? (A workable patch - maybe)

      OR

      Get LinkSys to check their disconnect logic? (Not likely)

      Start debugging lower layers? (Well, that's the same as the first idea (Not a satisfactory solution - and no guarantee of success.

      Guess I'll put in a response (in file) length test to validate UserAgent error - bleearghr ;)

      Unless someone can help me with the lower level stuff.

      {"Why don't you do it in Perl?" - Well, one has to start somewhere! ;))) }

      Steve Potter
      http://www3.sympatico.ca/steven.potter/