When you send a message over a socket, you have no control over how many pieces the message gets chopped up into. For instance, if you send the message 'hello world', that could get sent in 1 piece, 2 pieces, 3 pieces, etc., and the pieces could all be different lengths. Therefore, the sender and the receiver of the message have to agree on a signal for the end of the message--that way the receiver knows when it can stop trying to read from the socket.

Some protocols use end-of-file as the signal that the receiver should stop trying to read from the socket. The eof signal is sent when the other side closes the socket. In your case, amazon is doing a retry, so amazon doesn't close the socket, which means your while loop doesn't end after it reads the entirety of the message.

I don't think you can use the closing brace as a signal to tell you when you should stop trying to read from the socket, because closing braces can exist within a JSON formatted message as well. Maybe look for a line that consists of just a closing brace? Alternatively, you have to figure out what characters amazon sends between the end of the message and the retry, and look for those characters in your while loop so you can break out of it.

I looked around a little bit, and the retry feature is something people have been requesting, but nothing indicates that amazon has implemented retry yet or how to configure it. Your results indicate that amazon has indeed implemented that feature.

Also, note that when you use the line input operator, <>, you are not telling perl to read whatever data is available in the socket--instead you are telling perl to block and keep trying to read until perl reads in a newline. After reading a newline, the parts of the message that have been read up to that point are returned and the body of the while loop executes.

"instead you are telling perl to block and keep trying to read until perl reads in a newline. " OK, good point - thanks. How do I switch it so that I grab whatever's there all at once?

I didn't necessarily mean for you to change your line oriented input to something else, I just wanted you to know that the read will block and accumulate pieces of the message until it encounters a newline.

You can treat a socket as if it were a file. Ideally, amazon would close the socket after sending the message, which would send an eof signal into the socket, and just like when reading a file, your while loop would end. But based on what you say about the input you are receiving, amazon doesn't do that. You could read the whole file in one go, but then you would get the duplicated message in the retry and any subsequent retrys until amazon closed the socket.

So, you need to examine the input to see what characters lie between the end of the message and the retry. You should inspect() the string to see all the hidden characters. It may be as simple as testing for a line that contains only a closing brace and no whitespace, i.e. the final closing brace of the JSON message, and then breaking out of your while loop.

I just checked the JSON spec here, json, and whitespace before or after a brace is ignored, so valid JSON could look like this:

{
    "glossary": {
        "title": "example glossary"
}
    "book": {
        "name": "Hello World"
}
}
[download]

...which means a closing brace on a line by itself would not signal the end of the message. Whether amazon actually does that is another thing entirely.

use Plack::Request;
my $app_or_middleware = sub {
    my $env = shift; # PSGI env
    my $req = Plack::Request->new($env);
    my $path_info = $req->path_info;
    my $query     = $req->param('query');
    my $content   = $req->content;
    my $res = $req->new_response(200); # new Plack::Response
    $res->content_type('text/plain');
    $res->body([ "Received your request, all ",
$req->content_length,
" bytes, thanks , here Is what I got \n",
$content ]);
    return $res->finalize;
};

require Plack::Runner;
my $runner = Plack::Runner->new;
$runner->parse_options( qw' --host 127.0.0.1 ');
$runner->run($app_or_middleware);
[download]