LWP::UserAgent socket errors

steves has asked for the wisdom of the Perl Monks concerning the following question:

I have a script that walks through some sites of ours and buys products. LWP::UserAgent is used along with the other expected modules: HTTP::Request, HTTP::Response, HTTP::Headers, etc.

The problem is that periodically I get those self-generated 500 errors from the HTTP protocol layer. These are almost always EOF before status line seen.

Is this something others have hit? If so I'd be interested in hearing how you handled them. Right now it's too complicated for me to try and get the user back in without knowing the exact chain of responses, so I just bomb out and throw these to an exception pile. Ideally I'd like to recover and continue, but without a response that's not possible. I'm thinking maybe there's some socket level retry that's warranted or something, but I don't see how to do that at the LWP level.

Comment on LWP::UserAgent socket errors

Replies are listed 'Best First'.
Re: LWP::UserAgent socket errors by merlyn (Sage) on Oct 05, 2001 at 18:37 UTC
Something dropped the connection after you connected. Perhaps TCP wrappers or some other application-layer blocking system let the connection get established, only to find out they didn't want to talk with you. If it's only intermittent, that might mean an accept or fork failure on their part. -- Randal L. Schwartz, Perl hacker	[reply]
Re: LWP::UserAgent socket errors by Anonymous Monk on May 21, 2003 at 20:10 UTC
I have a similar (exasperating ;) ) problem while using LWP UserAgent to get a Status page from a LinkSys BEFSR41 (from LAN side peer client). AFTER full 4673 bytes of Status HTML is received from server, LWP sends a redundant terminal request that provokes a 500 "Can't read entity body" response AND wipes out the $response->content. Same thing happens if I use a filename method, except that file contains 4673 expected response bytes but $response->content is empty. LWP::DebugFile Log:- # LWP::DebugFile logging to lwp_3ecbccb9_1983.log # Time now: {1053543619} = Wed May 21 15:00:19 2003 # Time now: {1053543629} = Wed May 21 15:00:29 2003 LWP::UserAgent::new: () LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://192.168.1.1/Status.htm LWP::UserAgent::_need_proxy: Not proxied # Time now: {1053543633} = Wed May 21 15:00:33 2003 LWP::Protocol::http::request: () LWP::Protocol::collect: read 340 bytes LWP::UserAgent::request: Simple response: Unauthorized # Time now: {1053543634} = Wed May 21 15:00:34 2003 LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://192.168.1.1/Status.htm LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 903 bytes LWP::Protocol::collect: read 3770 bytes LWP::UserAgent::request: Simple response: Internal Server Error [download] What can I do to prevent final (redundant) request from destoying $response->content? Why doesn't $response->content retain accumulated response received? Did you find a solution to this Steve steven.potter@sympatico.ca	[reply] [d/l]