read from socket blocks until buffer is full

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello dear Monks,

I tried reading from a socket as one might from a file (I know, no error handling, that's not the point here):

my $buf;
my $read;
while(($read = $s->read( $buf, 1024 )) > 0) {
    my $chunk = substr $buf, $read;
    # process chunk
}
[download]

When the socket is not closed after the sender sent its stuff the read blocks until 1024 bytes are available. So if I don't know in advance how many data I'm about to receive, I have to use a buffer size of one. Is this correct? If yes, isn't this inefficient? If not, how would a better solution look like?

An example would be parsing an HTTP header. The length field is not always at the same position so until I find it, I have to take small steps in order not to try to read more bytes than are available in the current response. With keep-alive that socket stays open so no EOF there.

The main reason I ask is because I do a lot of Java programming. When you read from a socket in Java the read returns even if not enough data is received to fill the whole buffer in a short amount of time.

Comment on read from socket blocks until buffer is full Download Code

Replies are listed 'Best First'.
Re: read from socket blocks until buffer is full by ikegami (Patriarch) on Feb 27, 2011 at 19:47 UTC
You want `sysread` instead of `read`. `read` will read until the specified number of bytes has arrived. Say your header is 500 bytes long and there's no body (e.g. a GET request), `read` will wait for another 524 bytes more to come even though more won't come. `sysread` will return as soon as some bytes are available. `my $buf = ''; my $header; for (;;) { my $rv = sysread($sock, $buf, BLK_SIZE, length($buf)); die("Can't read: $!\n") if !defined($rv); die("Can't read: Premature EOF\n") if !$rv; if ($buf =~ s/^(.*\r\n\r\n)//s) { $header = $1; last; } }` [download] Use the same technique to read the body. Don't forget that some of the body may already be in $buf.	[reply] [d/l] [select]
Re^2: read from socket blocks until buffer is full by Anonymous Monk on Feb 28, 2011 at 18:13 UTC
Ok, this works, thank you. I never used `sysread` so I didn't think of that.	[reply] [d/l]
Re: read from socket blocks until buffer is full by Corion (Patriarch) on Feb 27, 2011 at 18:27 UTC
Maybe you want to set the sockets to nonblocking? Then you can read as much data as there is for the moment. Also consider looking at the IO (and event) multiplexing frameworks, like AnyEvent, POE and IO::Async. If you prefer to roll your own, IO::Select is a good basis for a start.	[reply]