Coruscate has asked for the wisdom of the Perl Monks concerning the following question:

Introduction

This is YACSQ (Yet Another Client/Server Question). After a long debate with myself, I have settled upon one of many options for how to go about setting up a Client/Server application. Getting data back and forth between the server and the client is complete (and of course is the easiest step IMO). Now I have stumbled upon what I believe will be the biggest challenge in the project: selecting a communication design (ie: the protocol).

What I have so far

The client/server setup I have chosen is quite simple indeed. The client listens on STDIN and on the socket simultaneously in order to create a non-blocking socket. The client can exchange data with the server even while the user is being slow at typing in the next line. It works excellent. The server loops through the clients and helps each one out as it sees fit. Great. But I don't know where to start to create the protocol that the client and server can use to say what data they are about to send.

What I've thought of

As another decision, I have decided to use sysread() and syswrite() for all communication between the two apps. Using this method means that newlines won't necessarily be the line terminator. As well, I can read as many bytes at a time as I wish (a good thing if I end up using a fixed-length protocol). I've thought of multiple ideas, none of which I'm sure I'd like to implement. I'm hoping someone has better ideas today than myself.

Idea #1: XML Tagged
Use an xml-based interface for communications. Every time the client or server wishes to say something, they pass along a block of text such as the below example. The server sends the first block to the client, requesting a password. The server then waits for a response.

# server sends to client <command name="get_passwd"> Please enter your password: </command> # client sending to server <command name="text_data"> example_password </command>


pros: looks clean, quite obvious as to what is going on.
cons: parsing the tags. Seems annoying. Help with other cons?

Idea #2: Fixed-Length Communication
Using a fixed-length protocol seems simple to me, so I must be missing something. I was thinking maybe of following the idea of the 'content-length' from HTTP. The example below captures my thoughts pretty well I think. The first two chars show the length of the command in 2-digit form ('get_passwd' is 10 chars long). Following the command is a 4-digit number specifying the length of the data to follow (I figure 9999 is the longest I'll ever send :P).

# server sends to client. syswrite( $client, "10get_passwd0027Please enter your password:" ); # client reads from server my ($len, $cmd, $data); sysread($server, $len, 2); sysread($server, $cmd, $len); sysread($server, $len, 4); sysread($server, $data, $len); # $cmd eq 'get_passwd' # this is # $data eq 'Please enter your password:' # correct, right?


pros: seems quite trivial. No parsing involved. Just multiple reads.
cons: can't think of any. Help?

So are my ideas out of this world in a bad way or do I have a good brainwave or two focused on this? Any insights would be greatly appreciated and welcomed with wide open arms. Slaps across the face for stupid thoughts are welcome as well :)


If the above content is missing any vital points or you feel that any of the information is misleading, incorrect or irrelevant, please feel free to downvote the post. At the same time, please reply to this node or /msg me to inform me as to what is wrong with the post, so that I may update the node to the best of my ability.

Replies are listed 'Best First'.
Re: Choosing a client/server protocol
by TVSET (Chaplain) on Jul 07, 2003 at 20:32 UTC
Re: Choosing a client/server protocol
by Thelonius (Priest) on Jul 07, 2003 at 21:06 UTC
    Contrary to sauoq, I think using length-prefixed strings is an excellent method, very simple and efficient. The only disadvantage is that you can't use Telnet to debug it.

    The BEEP protocol uses it, and Marshall Rose says it is the most efficient method. See RFC3117: On the Design of Application Protocols (a good read).

    Bernstein uses strings preceded by an delimited decimal length string in netstrings.

    Note that there are really several issues here which can be handled together or separately. First is the reader knowing how much data to expect in the next "chunk". Second, delimiting logical strings within a transaction. Third, knowing whose turn it is to talk. In your example, it appears issues one and two are combined, and the third issue is just handled by the client and server programs just knowing (e.g.) how many strings to accept. For simple protocols that's okay, but it's better to have some general method to indicate line turnaround (e.g. SMTP/POP error messages continue with a "-" as the fourth character in a line to indicate that there's more to come, and use a "." on a line by itself to indicate the end of a message.

Re: Choosing a client/server protocol
by sauoq (Abbot) on Jul 07, 2003 at 20:37 UTC

    It might help to know what the nature of the data is that you wish to exchange with this protocol...

    Going the XML route might indeed be a good idea. Can you leverage XML RPC or SOAP?

    Your second idea is likely to be much more error prone and brittle. I wouldn't go that route. You might consider something similar to it though, especially if you anticipate the need (for debugging or otherwise) to connect to your server with, say, telnet and communicate in the protocol by hand. You might take a look at SMTP, for example.

    -sauoq
    "My two cents aren't worth a dime.";
    

      The server will be sending seemingly random-length bodies of text ranging from 5 characters to 1000-2000 characters in length. The clients will be sending in short lines for the most part, possibly larger bodies less frequently.

      Might you be able to provide a reason or two as to why the fixed-length method would not be a good route to go? All communication will be a simple "this command" to handle "this data". I'd like to hear why it's a bad idea (or maybe just not the best).

      TVSET's link in his response contains examples of the SMTP, IMAP, and POP3 protocols. I'm in the process of combing through the pages right now.


      If the above content is missing any vital points or you feel that any of the information is misleading, incorrect or irrelevant, please feel free to downvote the post. At the same time, please reply to this node or /msg me to inform me as to what is wrong with the post, so that I may update the node to the best of my ability.

        Might you be able to provide a reason or two as to why the fixed-length method would not be a good route to go? All communication will be a simple "this command" to handle "this data". I'd like to hear why it's a bad idea (or maybe just not the best).

        Well, I was looking at it in the context of your example which seems to want to support direct access via telnet. Typing 09text_data0016example_password seemed painful. I see I may have misread your intentions though.

        Just the same, specifying a fixed number of digits for the data size does seem brittle. If you ever find yourself needing to support chunks of data with more 10,000 or more characters you'll have to change a bunch of code.

        Other than that, it's really fine. It makes a tradeoff between the need to know the length of your data before you send it and the need to process the stream looking for delimiters.

        I'd really suggest trying to use an existing protocol if possible though. It could save you a lot of time by allowing you to use modules which are already available and have been extensively used and tested.

        -sauoq
        "My two cents aren't worth a dime.";
        
Re: Choosing a client/server protocol
by waswas-fng (Curate) on Jul 07, 2003 at 21:28 UTC
    And depending what you are going to be doing with this, the use of bitwise flags and a bin stream may be worthwhile. if the chatter is constant and has a lot of states you can fit a ton of information into bit flags.

    -Waswas
Re: Choosing a client/server protocol
by zentara (Cardinal) on Jul 08, 2003 at 16:30 UTC
    I'm no expert at sockets, but have you looked at the way Net::EasyTCP sends data? It just uses a hash to send complex objects. Maybe you can gleam some clues.
    #Send and receive a simple string $client->send("HELLO THERE") || die "ERROR SENDING: $@\n"; $reply = $client->receive() || die "ERROR RECEIVING: $@\n"; print "$reply\n"; #Send and receive complex objects/strings/arrays/hashes by reference #%hash = ("to be or" => "not to be" , "just another" => "perl hacker") +; #$client->send(\%hash) || die "ERROR SENDING: $@\n"; #$reply = $client->receive() || die "ERROR RECEIVING: $@\n"; #foreach (keys %{$reply}) { #print "Received key: $_ = $reply->{$_}\n"; #} #Send and receive large binary data #for (1..4096) { #for (0..255) { #$largedata .= chr($_); #} #} #$client->send($largedata) || die "ERROR SENDING: $@\n"; #$reply = $client->receive() || die "ERROR RECEIVING: $@\n"; #$client->close();