yegg has asked for the wisdom of the Perl Monks concerning the following question:

I started seeing weird behavior from LWP (5.835) when accessing the Bing API. In particular, I'm doing something like this: $response = $ua->request(GET $url);

The problem is $response->content intermittently (~50% of the time) returns only a partial JSON response.

When that happens, the header reads like this:

HTTP/1.1 200 OK Connection: close Date: Sun, 24 Oct 2010 15:24:50 GMT Content-Length: 38511 Content-Type: application/json; charset=utf-8 Client-Aborted: die Client-Date: Sun, 24 Oct 2010 15:24:50 GMT Client-Peer: 72.246.30.144:80 Client-Response-Num: 1 X-Died: read failed: Connection reset by peer at /usr/local/lib/perl5/ +site_perl/5.8.9/LWP/Protocol/http.pm line 388.

I see the error in the X-Died header as the docs suggest. However, if I use curl/wget/lynx/whatever on the same machine I get the full response every time.

Any ideas?

Replies are listed 'Best First'.
Re: Partial content from LWP
by Corion (Patriarch) on Oct 24, 2010 at 15:29 UTC

    Investigate what the two clients do differently. Likely, you will need a network sniffer to see what actually goes over the wire.

      I've mimicked the headers from curl exactly, and still get the error. I can also replicate it using the lwp-request tool.

        So what difference does the network sniffer show?