http://qs1969.pair.com?node_id=1223514

So, your project is going fine, your codebase is groing fast. But now your have the problem that some of your processes have to communicate with each other. Maybe, some temperature sensor needs to report its sensor value every few seconds to the central heating system. Maybe the central heating system needs to know if the windows are open and close them before heating the house. Another process wants to count how many times the door has been opened and log the sum once a minute...

Net::Clacks to the rescue!

The Net::Clacks modules implement a client/server based interprocess messaging. Without going too much into the internals of the protocol, a client can either send notifications ("event xy has just happened") or values ("the reading for sensor xy is now 42"). Other clients may (or may not) choose to listen to those broadcasts.

Net::Clacks also implements Memcached-like value storage and retrieval. So instead of broadcasting, a value can be stored, read out, incremented, decremented and deleted.

A note on security: Currently, the system only implements a shared-secret type thing (all clients in a clacks network use the same username/password). This will get changed in the future. I'm planning to make it so that you can override the authentication checks with your own function and return which permissions the client has. But that is not yet implemented.

Let's do a simple example project: Server, chatclient, chatbot and a clock process to trigger some actions at the start of every minute.

First of all, we need a server. For this, we need an XML config file and a bit of Perl code.

#!/usr/bin/env perl #---AUTOPRAGMASTART--- use 5.020; use strict; use warnings; use diagnostics; use mro 'c3'; use English; use Carp; our $VERSION = 4.1; use Fatal qw( close ); use Array::Contains; #---AUTOPRAGMAEND--- my $isDebugging = 0; if(defined($ARGV[1]) && $ARGV[1] eq "--debug") { $isDebugging = 1; } use Net::Clacks::Server; my $configfile = shift @ARGV; croak("No Config file parameter") if(!defined($configfile) || $configf +ile eq ''); my $worker = Net::Clacks::Server->new($isDebugging, $configfile); $worker->init; $worker->run;

And the config (saved as clacks_master.xml, we'll add a slaveserver later on):

<clacks> <appname>Clacks Master</appname> <ip>127.0.0.1</ip> <port>18888</port> <pingtimeout>600</pingtimeout> <interclackspingtimeout>10</interclackspingtimeout> <ssl> <cert>exampleserver.crt</cert> <key>exampleserver.key</key> </ssl> <username>exampleuser</username> <password>unsafepassword</password> <throttle> <maxsleep>5000</maxsleep> <step>1</step> </throttle> <hosts> <developmentbox> <ip>128.0.0.1</ip> <ip>::1</ip> </developmentbox> </hosts> </clacks>

Oh, and yes, we need an OpenSSL certificate as well. For this demo we will make a self signed one. Not very secure, but sufficient for a proof of concept:

openssl req -new -newkey rsa:4096 -x509 -sha256 -days 36500 -nodes -ou +t exampleserver.crt -keyout exampleserver.key

Now let's start the server:

perl server.pl clacks_master.xml

First client will be the "clock.pl" process, this is basically a "write only" thing, it only sends a notify() to all clients listening to it at the start of every minute.

#!/usr/bin/env perl #---AUTOPRAGMASTART--- use 5.020; use strict; use warnings; use diagnostics; use mro 'c3'; use English; use Carp; our $VERSION = 4.1; use Fatal qw( close ); use Array::Contains; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Term::ReadKey; use Time::HiRes qw(sleep); use Data::Dumper; my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'clock'; my $is_caching = 0; my $chat = Net::Clacks::Client->new('127.0.0.1', 18888, $username, $pa +ssword, $applicationname, $is_caching); my $clockname = 'example::notify'; my $last = ''; while(1) { my $now = getCurrentMinute(); if($now ne $last) { $chat->notify($clockname); $chat->ping(); $last = $now; } $chat->doNetwork(); while((my $msg = $chat->getNext())) { if($msg->{type} eq 'disconnect') { print '+++ Disconnected by server, reason given: ', $msg-> +{data}, "\n"; last; } } sleep(0.2); } sub getCurrentMinute { my ($sec,$min, $hour, $mday,$mon, $year, $wday,$yday, $isdst) = lo +caltime time; $year += 1900; $mon += 1; $mon = doFPad($mon, 2); $mday = doFPad($mday, 2); $hour = doFPad($hour, 2); $min = doFPad($min, 2); return "$year-$mon-$mday $hour:$min"; } sub doFPad { my ($val, $len) = @_; while(length($val) < $len) { $val = '0' . $val; } return $val; }

Notice, we also have to send the occasional ping(). This tells the server we are still alive, so it doesn't close the connection. You usually do this in your main loop. I personally had a bug in which the client kept sending some data while it was stuck in a subroutine, so this is some sort of insurance that the clients higher brain functions still get called...

Another thing to note is the doNetwork() call. Message sending and receiving over the network doesn't happen automatically, messages get buffered in the client library. This reduces network traffic when handling lots of small messages by bundling them into bigger packets.

Now, let's do the actual chat client. This is quite similar, except we listen() to some messages at the beginning and also handle the extra message types. Oh, and of course, take line-by-line user input and send those chat messages to the server.

#!/usr/bin/env perl #---AUTOPRAGMASTART--- use 5.020; use strict; use warnings; use diagnostics; use mro 'c3'; use English; use Carp; our $VERSION = 4.1; use Fatal qw( close ); use Array::Contains; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Term::ReadKey; use Time::HiRes qw(sleep); use Data::Dumper; my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'chatclient'; my $is_caching = 0; my $chat = Net::Clacks::Client->new('127.0.0.1', 18888, $username, $pa +ssword, $applicationname, $is_caching); #print 'Connected to server. Info given: ', $chat->getServerinfo(), "\ +n"; my $chatname = 'example::chat'; my $clockname = 'example::notify'; $chat->listen($chatname); $chat->listen($clockname); $chat->ping(); $chat->doNetwork(); my $nextping = time + 60; while(1) { my $line = ReadLine -1; if(defined($line)) { chomp $line; if(length($line)) { last if(uc $line eq 'QUIT' || uc $line eq 'EXIT'); $chat->set($chatname, $line); } } if($nextping < time) { $chat->ping(); $nextping = time + 60; } $chat->doNetwork(); while((my $msg = $chat->getNext())) { if($msg->{type} eq 'set' && $msg->{name} eq $chatname) { print '>>> ', $msg->{data}, "\n"; } elsif($msg->{type} eq 'notify' && $msg->{name} eq $clockname +) { print "*** Another minute has passed ***\n"; } elsif($msg->{type} eq 'disconnect') { print '+++ Disconnected by server, reason given: ', $msg-> +{data}, "\n"; last; } } sleep(0.2); }

At the beginning, we listen() to the two message names. On user input, we send it as a chat message to everyone else listening to the chat. Roughly every 60 seconds we ping() the server (keepalive again). Then we send and receive with doNetwork() and check to see if we got any interesting messages.

We could simplify the checks quite a bit since there are only two well defined messages (one 'notify', one 'set'), but i implemented the full checking here to illustrate the principle.

Last but not least, let's implement the most brain-dead chatbot we can think of. It repeats messages containing "simon" with "Simon says". It also implements a counter in Clacks for counting chat messages. Every time it gets a clock notification, it sends the number of chat messages seen since the last notification as a chat message and decreases the counter accordingly. (I could just set it to zero, but using decrement with the previously read out value is the correct thing to do, especially in cases where the counter is increased by some other program).

#!/usr/bin/env perl #---AUTOPRAGMASTART--- use 5.020; use strict; use warnings; use diagnostics; use mro 'c3'; use English; use Carp; our $VERSION = 4.1; use Fatal qw( close ); use Array::Contains; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Term::ReadKey; use Time::HiRes qw(sleep); use Data::Dumper; my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'chatbot'; my $is_caching = 0; my $chat = Net::Clacks::Client->new('127.0.0.1', 18888, $username, $pa +ssword, $applicationname, $is_caching); #print 'Connected to server. Info given: ', $chat->getServerinfo(), "\ +n"; my $chatname = 'example::chat'; my $clockname = 'example::notify'; my $countname = 'chatbot::linecount'; $chat->listen($chatname); $chat->listen($clockname); $chat->ping(); $chat->doNetwork(); my $nextping = time + 60; while(1) { my $line = ReadLine -1; if(defined($line)) { chomp $line; if(length($line)) { last if(uc $line eq 'QUIT' || uc $line eq 'EXIT'); $chat->set($chatname, $line); } } if($nextping < time) { $chat->ping(); $nextping = time + 60; } $chat->doNetwork(); while((my $msg = $chat->getNext())) { if($msg->{type} eq 'set' && $msg->{name} eq $chatname) { # Increment count for every chat message $chat->increment($countname); # very non-AI answer implementation of "simon says" if($msg->{data} =~ /simon/i) { $chat->set($chatname, 'RoboSimon says: ' . $msg->{data +}); $chat->increment($countname); } } elsif($msg->{type} eq 'notify' && $msg->{name} eq $clockname +) { # Every minute, retrieve our current count, send a chat me +ssage and decrease line count accordingly my $linecount = $chat->retrieve($countname); if(!defined($linecount)) { $linecount = 0; $chat->store($countname, 0); } $chat->set($chatname, $linecount . ' new chat messages dur +ing the last minute'); $chat->decrement($countname, $linecount); } elsif($msg->{type} eq 'disconnect') { print '+++ Disconnected by server, reason given: ', $msg-> +{data}, "\n"; last; } } $chat->doNetwork(); sleep(1); }

Ok, sure, this chat system will never get much users, because it just doesn't have good features. It's just a demo of some of the Net::Clacks capabilities.

But "what if" you actually get more users or processes that a single service or server can handle? No problem here, either. Net::Clacks::Server has what is called an "Interclacks" mode, which can be used to run in master/slave mode. When a slave starts up and connects (or reconnects after a network error) to the master, it drops it's locally stored variables and resyncs from the master.

After that, every message or stored variable you set/send/notify gets automatically synced to the master. A slave can also work as a master node at the same time, so you are able to chain nodes in a tree like pattern if you really so desire.

So, lets make a slave node. In our case, it will run on the same host with a different port (but you can trivially change it to run on a different server). All we need is an additional XML config file:

<clacks> <appname>Clacks Master</appname> <ip>127.0.0.1</ip> <port>18889</port> <pingtimeout>600</pingtimeout> <interclackspingtimeout>10</interclackspingtimeout> <ssl> <cert>exampleserver.crt</cert> <key>exampleserver.key</key> </ssl> <username>exampleuser</username> <password>unsafepassword</password> <throttle> <maxsleep>5000</maxsleep> <step>1</step> </throttle> <master> <ip>127.0.0.1</ip> <port>18888</port> </master> </clacks>

And then start it:

perl server.pl clacks_slaveserver.xml

So far, i have had Net::Clacks in use on a couple of live systems for over 2 years, most of them running 1000+ clacks connections at the same time. At least one of them having a constant 24/7 message rate of roughly 5000 messages per minute. Pretty much no crashes in the last few months. Obviously, you need to test, test, test before you decide to use it for your own systems.

"For me, programming in Perl is like my cooking. The result may not always taste nice, but it's quick, painless and it get's food on the table."