Re^2: unbuffered read from socket

I do think reading one character at a time is your best bet

It's generally poor form and very slow to read one char at a time with sysread(); it's unbuffered, as is C's read(), so you're making a system call per character. Ouch. If you ask for 1024 bytes and only one is available, it will be returned in $buf, sysread returning the number of chars actually read. For example:

my $buf; my $nread;
open(my $filehandle, '<', 'f.tmp') or die "error open: $!";
while ($nread = sysread($filehandle, $buf, 1024)) {
    print "nread=$nread buf='$buf'\n";
}
[download]

To illustrate my point, this program:

use strict;
my $buf; my $nread;
open(my $filehandle, '<', 'f.tmp') or die "error open f.tmp: $!";
binmode($filehandle);
open(my $fhout, '>', 'g.tmp') or die "error open g.tmp: $!";
binmode($fhout);
while ($nread = sysread($filehandle, $buf, 1)) {
    print $fhout $buf;
}
close($filehandle); close($fhout);
[download]

takes 27 seconds to copy file f.tmp to g.tmp, where f.tmp is perl-5.8.6.tar.gz (about 12 MB). If you simply change:

sysread($filehandle, $buf, 1)
[download]

above to:

sysread($filehandle, $buf, 1024)
[download]

the time reduces to 0.2 seconds.

Comment on Re^2: unbuffered read from socket Select or Download Code

Replies are listed 'Best First'.
Re^3: unbuffered read from socket by Errto (Vicar) on Dec 29, 2004 at 04:02 UTC
Yes, that is certainly true, but it does increase complexity because the developer then has to do his own buffering. IOW, he only wants one line but he might well end up reading more than that, up to 1k so that has to be stored somewhere.	[reply]
Re^4: unbuffered read from socket by eyepopslikeamosquito (Archbishop) on Dec 29, 2004 at 04:35 UTC
Good point Errto, you drove me to dig out the Perl Cookbook recipe 7.23, which, to my horror, also does a sysread() of one character at a time (and with no performance caveat). I wonder if anyone has encapsulated a nice clean buffering scheme to work in harness with select() and sysread()? Update: quick look on CPAN found nothing, but Lincoln Stein's book "Network Programming with Perl" (Chapter 13) features an IO::Getline module that seems to be what I'm after, namely "line-oriented reading from sockets/handles with access to internal buffer". One important point made in this recipe is that you must use unbuffered sysread() (and not the buffered I/O functions) with select() because select's response does not reflect user level buffering. Update: Found this interesting bit of history from comp.lang.perl.moderated (15-Jan-1999) describing a mythical sysreadline() function requested by Larry but, alas, never implemented.	[reply]
Re^5: unbuffered read from socket by Errto (Vicar) on Dec 29, 2004 at 05:03 UTC
It certainly sounds like the sort of thing you'd want a generic module for. That said, in this specific case Net::IRC seems to already exist. Actually, I'll mention that in my reply node in case the OP doesn't notice this one.	[reply]
Re^5: unbuffered read from socket by Forsaken (Friar) on Dec 29, 2004 at 11:18 UTC
my my, nice piece of history there indeed. glad i'm not the only one to think this is cumbersome. if even Larry agrees... :-) I have yet to find this illusive IO::Getline anywhere, do let me know if if turns up somewhere please.	[reply]