It looks like the remote end is sending a Unicode BOM (byte order mark) at the beginning of the stream, at a guess. Which is a bit naughty, because the headers didn't specify that it was going to be a UTF-8 stream. However this might be unrelated to the bug.

Take that ethereal dump, see if you can work it into a test case for LWP, and submit it as a bug to the maintainer. Make sure you include a working one for comparison, too.

This might involve a simple Test server, if you can't figure out how to test just the response processing component; this little script can do the trick; it takes two arguments: the response file to issue, and where to save the request that it read.

use IO::All;

my $response = shift;
my $target = shift;

my $server = io(":9207");
$server->autoclose(0);
my $sock = $server->accept;

$sock->autoclose(0);

my $doc;
while (my $line = $sock->readline) {
    last if $line =~ m/\A\r?\n\Z/;
    $doc .= $line;
}

{ $doc > io($target)->assert; }
$sock->print(io($response)->slurp);
$sock->shutdown;

# and it was over ... before it began
exit(0);
[download]

$h=$ENV{HOME};my@q=split/\n\n/,`cat $h/.quotes`;$s="$h/."
."signature";$t=`cat $s`;print$t,"\n",$q[rand($#q)],"\n";
[download]