So, Shall I dispense with the UserAgent Request and run it through lower level code? >:-{ (Defeating the purpose of UserAgent) (Not a satisfactory solution)

OR

Use the filename method and use a 'length of file (= length of response)' test before assuming that UserAgent error response is valid (i.e. rebuild the response->content from the collected file copy if response is not 'too short'? (A workable patch - maybe)

OR

Get LinkSys to check their disconnect logic? (Not likely)

Start debugging lower layers? (Well, that's the same as the first idea (Not a satisfactory solution - and no guarantee of success.

Guess I'll put in a response (in file) length test to validate UserAgent error - bleearghr ;)

Unless someone can help me with the lower level stuff.

Here are some things you could try:

1. export PERL_LWP_USE_HTTP_10=1 before you run Perl. It probably won't change anything, but you never know.
2. Try this:

   $ua = LWP::UserAgent->new(use_eval => 0);
   ...
   eval {
     $response = $ua->request($request);
   };
   if ($@) {
     if ($@ =~ /Connection refused/) {
       # ignore error
     } else {
       # just like, ya know, die or something
     }
   }
   # I think $response will still be intact here.
   [download]

   Very much untested!!!!
3. Upgrade Linksys firmware. There is one version on their web site, but the only thing it says it fixes is a UPnP error, so it probably won't help.
4. Modify the function my_read in Net/HTTP/Methods.pm to ignore ECONNRESET. I think this is the right place.

I tried:-


$ua = LWP::UserAgent->new(use_eval => 0);
.
.
eval {
  $response = $ua->request($request);
};
if ($@) {
  #print "eval: " . $@ . "\n";
  if ($@ =~ /Connection refused/) {
    print "Response Content...\n";
    print $response->content . "\n";
  } else {
.
.
[download]

It still didn't work:-


(print "eval: " . $@ . "\n";)
[download]

gives:-

eval: Can't read entity body: Connection reset by peer at /usr/local/l
+ib/perl5/site_perl/5.005/LWP/Protocol/http.pm line 337.
[download]

So, (looking at http.pm) there's the problem confirmed.
1) Linksys BEFSR41 drops the connection before HTTP layer is satisfied that data collection is complete.
2) HTTP layer destroys the response->content collected thus far. >:{

So, anyone care to tell me how to fix the HTTP layer so it does NOT destroy response->content on a dropped connection - if it already collected (at least some) data from current session?
OR
Do I resort to using filename method, with a "length of response" test, to validate UserAgent error, AND rebuild my $response->content from file? (bleaaurh).

Steve Potter
http://www3.sympatico.ca/steven.potter/

Oops, in my code I had /Connection refused/ it should be /Connection reset/. See if that works.

When I was at home, I didn't have any problems accessing the LinkSys (which is the same model). My firmware was from May 2002. I can't remember the version number. You might try upgrading the firmware.

Cheers
Ben.

OK, (and thanks for responding)
Here're the relavant code fragments.

(The credentials are fine. I know because if I use the filename method I get the expected response in the file).

Note: 903 bytes + 3770 bytes = 4673 bytes = size of "rspfile" if I use filename method)

(Also, though not coded in this fragment, if I print $response->content before testing for error - it appears empty)

I suspect, after digging a bit, that socket or HTTP layer is not happy with the LinkSys dropping the connection before sending an EOF or something of the sort.

I could drop the

$response = $ua->request($request.......);
[download]

and go to a lower level or I could use the filename method and read the legitimate content back in - but I'd rather fix the UserAgent than patch it! ;)



.
.
use AppConfig;
use File::Basename;
use Getopt::Long;
use HTML::Form;
use HTTP::Request::Common;
use HTML::TreeBuilder;
use LWP::DebugFile ('+');
use LWP::UserAgent;
use Pod::Usage;
use URI::file;
.
.
$ua = LWP::UserAgent->new();
$ua->agent('Mozilla/4.0');
$ua->max_size('5000');
.
.
$ua->credentials($netloc, $realm, $userid, $passwd);
.
.
$request = HTTP::Request->new(GET => "http://192.168.1.1/Status.htm");

$response = $ua->request($request);
#$response = $ua->request($request,"rspfile.html");

if ($response->is_error) {
        my $string;

        print "Error: " . $response->status_line . "\n";
        print $response->error_as_HTML;

        $string = $response->as_string;
        die "$string\n";
    }

.
.
----------------

FreeBSD/i386 (blah blah blah)

login: steve
Password:
Last login: Wed May 21 16:59:51 from 192.168.1.1
Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994
        The Regents of the University of California.  All rights reser
+ved.

FreeBSD 4.7-RELEASE (ACCESS) #0: Tue Feb 11 19:33:55 EST 2003

        Welcome blah blah blah

steve> cd perlstuff
steve/perlstuff> ./LinkSys.pl
getting http://192.168.1.1/Status.htm...
Error: 500 Can't read entity body: Connection reset by peer
<HTML>
<HEAD><TITLE>An Error Occurred</TITLE></HEAD>
<BODY>
<H1>An Error Occurred</H1>
500 Can't read entity body: Connection reset by peer
</BODY>
</HTML>

500 (Internal Server Error) Can't read entity body: Connection reset b
+y peer
Client-Date: Thu, 22 May 2003 12:43:11 GMT

-----------------
Log:-

# LWP::DebugFile logging to lwp_3eccc413_2a9d.log
# Time now: {1053606945} = Thu May 22 08:35:45 2003
# Time now: {1053606959} = Thu May 22 08:35:59 2003
LWP::UserAgent::new: ()
# Time now: {1053606960} = Thu May 22 08:36:00 2003
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://192.168.1.1/Status.htm
LWP::UserAgent::_need_proxy: Not proxied
# Time now: {1053606965} = Thu May 22 08:36:05 2003
LWP::Protocol::http::request: ()
# Time now: {1053606966} = Thu May 22 08:36:06 2003
LWP::Protocol::collect: read 340 bytes
LWP::UserAgent::request: Simple response: Unauthorized
# Time now: {1053606967} = Thu May 22 08:36:07 2003
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://192.168.1.1/Status.htm
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 903 bytes
LWP::Protocol::collect: read 3770 bytes
LWP::UserAgent::request: Simple response: Internal Server Error
ttyp0://access/home/steve/perlstuff>
[download]

Steve Potter http://www3.sympatico.ca/steven.potter/

I don't think this is an LWP::UserAgent problem. It may be something specific to the Linksys "web server". I can grab the status page from my Netgear router with no problem using this code:

#!/usr/local/bin/perl  

use strict;
use warnings;
$| = 1;

use LWP::UserAgent;
use HTTP::Request::Common qw(GET POST);
use HTTP::Cookies;

my $user = 'user';
my $pass = 'pass';
my $cookie_jar = HTTP::Cookies->new(file => '/tmp/cookie_jar', AutoSav
+e => 1);

my $ua = LWP::UserAgent->new();
$ua->cookie_jar($cookie_jar);
my $req = GET 'http://192.168.0.1/mtenSysStatistics.html';
$req->authorization_basic($user, $pass);
    
my $response = $ua->request($req);
print $response->code . ' ' . $response->message . "\n";
    
my $content = $response->content;
if ($content)
{
    print $content;
}
else
{
    print 'No content!';
}
print "\n";
[download]

Can you give this a try (after adjusting for your environment) and see if you still get the 500 error?

I added the cookie jar (relavant code below)
(I didn't spot any other functional difference)

Same Result :{

So, it looks like I'll have to dispense with the


$response = $ua->request($request);
[download]

and replace it with low level code - merde! ;)



use AppConfig;
use File::Basename;
use Getopt::Long;
use HTML::Form;
use HTTP::Request::Common;
use HTML::TreeBuilder;
use HTTP::Cookies;
use LWP::DebugFile ('+');
use LWP::UserAgent;
use Pod::Usage;
use URI::file;
.
.
my $cookie_jar = HTTP::Cookies->new(file => '/tmp/cookie_jar', AutoSav
+e => 1);
my $ua;
.
.
$ua = LWP::UserAgent->new();
$ua->agent('Mozilla/4.0');
$ua->max_size('5000');
$ua->cookie_jar($cookie_jar);
.
.
$ua->credentials($netloc, $realm, $userid, $passwd);
.
.
$request = HTTP::Request->new(GET => "http://192.168.1.1/Status.htm");

$response = $ua->request($request);
#$response = $ua->request($request,"rspfile.html");

if ($response->is_error) {
        my $string;

        print "Error: " . $response->status_line . "\n";
        print $response->error_as_HTML;

        $string = $response->as_string;
        die "$string\n";
    }
.
.
[download]


Steve Potter
http://www3.sympatico.ca/steven.potter/

No, it is LWP that is giving the error "Can't read entity body". That is from LWP/Protocol/http.pm (line 338 in the version I have). It is giving that error because the LinkSys reset the socket connection instead of closing it (or at least LWP thinks that's what happened). It could be a bug in LWP or the FreeBSD TCP/IP layer, but it could also be a bug in the LinkSys (somewhat more likely, I think).

So, Shall I dispense with the UserAgent Request and run it through lower level code? >:-{ (Defeating the purpose of UserAgent) (Not a satisfactory solution)

OR

Use the filename method and use a 'length of file (= length of response)' test before assuming that UserAgent error response is valid (i.e. rebuild the response->content from the collected file copy if response is not 'too short'? (A workable patch - maybe)

OR

Get LinkSys to check their disconnect logic? (Not likely)

Start debugging lower layers? (Well, that's the same as the first idea (Not a satisfactory solution - and no guarantee of success.

Guess I'll put in a response (in file) length test to validate UserAgent error - bleearghr ;)

Unless someone can help me with the lower level stuff.

{"Why don't you do it in Perl?" - Well, one has to start somewhere! ;))) }

Steve Potter
http://www3.sympatico.ca/steven.potter/

It's because I needed to supply credentials that I resorted to using LWP ;)

Yes, the credentials are fine. I know because I get the expected response in the rspfile when I use the filename method.
Whether I use filename method or not I get the "500 Can't read entity body..." reported back and the $response->content does NOT contain the collected server response prior to the error.

In filename method, though, the rspfile is fine and $response is not.

Steve Potter http://www3.sympatico.ca/steven.potter/