in reply to Re^7: sysread and syswrite in tcp sockets
in thread sysread and syswrite in tcp sockets

OK. When you say persistent I assume that means that a number of separate chunks of data are going to be sent. What you need is a way to signal to the client the end of each chunk of data.

As indicated elsewhere, one way is to prefix each chunk with its length -- and do two simple reads per chunk, one for the length and one for the actual chunk -- ie an in-band signal of the end of each chunk. (If the sender closed the connection after each chunk, that would be an effective out-of-band signal.) (For completeness: if you cannot tell how long each chunk is before starting to send, you could break it into sub-chunks, preceed each one by its length and send an empty sub-chunk at the end.)

However, as this connection will persist for some time, then the possibility of getting a network error (or a machine dieing or being turned off or some other disturbance) increases. So I would go back to the (slightly) more involved sysread and deal with the possibility of errors.

Now you have two other possibilities:

  1. in-band: if the data allows it, send an "end-of-chunk" marker between each chunk. Now the reader needs to check after each sysread for the marker. Noting that the marker may not be at the end of the result of each sysread, though it probably will be.

  2. out-of-band: time -- ensure that the sender will send each chunk in a rapid sequence of packets (so no more than 's' seconds between packets), and then wait at least 't' seconds before sending the next chunk. Now, once a chunk has started, the reader concludes that a chunk is complete if it has to wait more than 'r' seconds for further data. To cope with variable delays introduced by the system, you'll want the time 's' to be significantly less than 'r', which in turn is significantly less than 't' -- where what is significant depends on your system (network and client and server) and the application (how long 't' is and how long you can make 'r' without violating any latency requirement).

You say you've achieved this using C# quite readily. I guess it is implicitly doing (2) above, where 's' is small -- perhaps because each chunk is not very big and is sent in a single Send and the system is lightly loaded. So 'r' can be quite small. (And 't' is implicitly bigger than 'r'.)

Having said all that, using time as the end-of-chunk signal can fail. If the system becomes busy during the transmission of a chunk, delaying a packet for 'r' seconds or more, the receiver will be fooled into thinking that the chunk is complete, and will treat the rest of the chunk as the start of another -- giving you two broken chunks. If the sender (or the network between receiver and sender) fails part way through a chunk the receiver will also be fooled. So... we're in the realm of what is reliable enough... if and 's' is very small and 'r' is small, and the system is generally reliable, and the chunks are short (so failures and delays are unlikely to interrupt a chunk in progress), then this may be good enough. But, unless you have some other way of detecting broken chunks, there remains the danger of a "silent" error.

You ask for "reliable". If you want 100% reliable, you need any one of the in-band end-of-chunk signals, or some other way of detecting broken chunks. Otherwise, subject to all the caveats above, you can use time as the signal -- having verified for yourself, on your system, that it is reliable enough for your needs.

Replies are listed 'Best First'.
Re^9: sysread and syswrite in tcp sockets
by rustybar (Novice) on Jan 09, 2009 at 08:08 UTC

    Hi, thanks for all the help there.

    From the comments you all have given, i reckon that the best way to receive data is to have have it's size known in the first place

    However, in my case it's not desirable to add more bytes to indicate the size of data. I know 4 bytes isn't alot, but i'm consider maximum cost saving of bandwidth.

    Thus, I guess the alternative way is to use time as the indicator to separate the different data together.

    Thanks for all the input. I will test out further to see what I can get

      I know 4 bytes isn't alot, but i'm consider maximum cost saving of bandwidth.... the alternative way is to use time as the indicator

      That doesn't make a lot of sense. All the time you spend waiting is equally wasted badwidth. Use or lose it.

      If you're thinking about the cost of those extra 4-bytes, you said the file sizes are 10.5k, which means those 4 bytes represent 0.009% of your traffic. How much can that be?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.