jhanna has asked for the wisdom of the Perl Monks concerning the following question:

ASSP.sf.net is an Anti-Spam SMTP Proxy written in perl. It uses IO::Socket::INET and IO::Select to multiplex socket connections from SMTP clients and to the mail transport.

For some reason it's dropping bytes. It's not in my perl code, because it happens at irregular intervals, but one user found this pattern:

There is a pattern to the first bad byte, but I haven't figured it out
yet...

The pattern seems to be based on a 285K byte (291840) cycle

After the first 57 bytes lost the system will:
successfully transfer 291783 bytes 
loose another 57
successfully transfer 291783 bytes 
loose another 57
successfully transfer 291783 bytes 
loose another 57
Etc...etc....

Test .125-meg.mpeg           128000
              Verified OK

Test .25-meg.mpeg            255943
              Missing Bytes, Total Len: 255943
missing: 57  Bytes at Location  141246 
              Verification Failed

Test .5-meg.mpeg             511886
              Missing Bytes, Total Len: 511886
missing: 57  Bytes at Location  187758 
missing: 57  Bytes at Location  479541 
              Verification Failed

Test 1-meg.mpeg              1023829
              Missing Bytes, Total Len: 1023829
missing: 57  Bytes at Location  207651 
missing: 57  Bytes at Location  499434 
missing: 57  Bytes at Location  791217 
              Verification Failed

My question is if there is a socket setting that will improve buffering or provide some other means of making the sockets more robust.

Thanks,
John

update (broquaint): title change (was socket problem)

Replies are listed 'Best First'.
Re: socket periodically loses data
by jhanna (Scribe) on Feb 13, 2003 at 20:54 UTC
    here's the code in question... Maybe there's a more effecient way, but I think this aughta work...

       $buf='';
   if($fh->sysread($buf,10240)>0) {
    $this=$Con{$fh};
    $buf=$this->{_}.$buf;
    while($buf=~/(.*\n)/g) {
     ${$Con{$fh}}{getline}($fh,$1);
     last unless $Con{$fh};  # it's possible that the connection can b
+e deleted while there's still something in the buffer
    }
    if($Con{$fh}) {
     ($this->{_})=$buf=~/([^\n\r]*)$/s;
     if(length($this->{_}) > 10000) {
      ${$Con{$fh}}{getline}($fh,$this->{_});
      ($this->{_})='';
     }
    }
   } else {
    done($fh);
   }

    [download]
[reply]
[d/l]
      Probably nobody cares, but this was the fix: 
         $buf='';
   if($fh->sysread($buf,4096)>0) {
    d(2);
    $this=$Con{$fh};
    $buf=$this->{_}.$buf;
    $bn=$lbn=-1;
    while(($bn=index($buf,"\n",$bn+1)) >= 0) {
     $s=substr($buf,$lbn+1,$bn-$lbn);
     #print DEBUG  "doing <$2>\n" if $DEBUG;
     ${$Con{$fh}}{getline}($fh,$s);
     last unless $Con{$fh};  # it's possible that the connection can b
+e deleted while there's still something in the buffer
     $lbn=$bn;
    }
    d(3);
    if($Con{$fh}) {
     ($this->{_})=substr($buf,$lbn+1);
     if(length($this->{_}) > 10000) {
      d(4);
      ${$Con{$fh}}{getline}($fh,$this->{_});
      ($this->{_})='';
     }
    }
   } else {
    d(5);
    done($fh);
   }

      [download]
[reply]
[d/l]
Re: socket periodically loses data
by extremely (Priest) on Feb 13, 2003 at 19:43 UTC
    Yah know, 291783 / 57 is 5119. So you are loosing 1 chunk out of 5120. That looks magic number-y to me.

    --
    $you = new YOU;
    honk() if $you->love(perl)

[reply]
Re: socket periodically loses data
by jasonk (Parson) on Feb 13, 2003 at 18:51 UTC

    Why do you believe that being predictable means it isn't a code problem? TCP/IP is supposed to ensure packet delivery, if your perl code isn't broken, then your network stack is.

[reply]
      I'd totally love to find someone prove me wrong (the code can be downloaded from the sourceforge site), but I'm baffled how my code which just gets lines from the socket and processes them would work for 99.99% of the lines and fail on the 0.01% -- it's doing the same thing over and over again. Maybe it is the TCP/IP stack, but then it's broken for Linux and Windows... I'd think someone would have noticed.

      john

[reply]

Back to Seekers of Perl Wisdom