IPC (inter-process communication) is a large part of most application i maintain, and so is in-memory caching (key/value store). To do this in a way i like, i created Net::Clacks. In fact, in 2018 i wrote a simple tutorial on PerlMonks: Interprocess messaging with Net::Clacks, but i guess a more up-to-date version could be useful for someone out there.
Before we start with the code, i'll have to explain a few concepts. The protocol has two different way to handle data. The first thing is "somewhat real time" messaging, the other is to provide an in-memory key/value store. Everything is based on, what boils to, named variables. You can name the variables pretty much anything, as long as they don't contain spaces and equal signs. Altough i highly suggest you avoid Unicode, as this not not well tested. In my programs, i usually use "fake perl classnames" ("PerlMonks::Chatterbot::messagecount"), but that's just a convention for my own stuff. You can be as boring or creative as you like with this stuff.
Real-time messages come in two different flavours: NOTIFY ("some event has happened") and SET ("this is the new value for a thing"). To receive those messages from the server, you have to LISTEN to them.
The key/value part of this is just like what it says on the label. You can STORE, RETRIEVE, REMOVE, INCREMENT and DECREMENT them. Other clients will not get automatically notified, you will have to do that yourself. One exception (for the case i use most) is the SETANDSTORE command, which stores a variable and broadcasts it to other clients, all in one go.
Most commands are fire-and-forget commands (the client doesn't have to wait for the server to process them). The exception is where the clients actively retrieves data from the server.
To optimize network efficiency, commands are buffered in memory. You have to call doNetwork() on the client side to trigger a send/receive cycle.
To better check for timeouts (including clients that don't disconnect but hang in some other way), the client has to regularly send PING commands, but it can temporarily disable this requirement by sending a NOPING command.
If you have to handle a lot of clients, or clients on multiple servers, you can run clacks servers in master/slave mode. This uses an expanded set of commands to implement what i call "interclacks mode", which includes cache sync and simple time offset calculations. But this is a bit outside this tutorial...
The last thing to note is that the clacks server can use a persistance file for the in-memory key/value store. So it can (mostly) restart-protect the data. This isn't perfectly real-time, so if you store a value and then immediately kill the server, those changes may or may not be available when the server is started again.
It's also possible to configure a few caching timeframes, that say how often stale entries are purged from the key/value store, how long entries are kept if they are not accessed in any way (default=1 day) and how long the server(s) remember that a key was deleted (for the purpose of interclacks resync). But again, advanced topic. I recommend leaving the defaults unless you run into problems.
Net::Clacks requires at least Perl 5.36 at the time of writing, because a) i switched to sub-signatures and b) i generally don't support completely outdated Perl versions. (If you want to run Net::Clacks on a completely outdated RedHat box, you might have to add some money to RedHats pay-to-win "Enterprise" racket so they can make you a broken, backported version.)
(Info for newbies to PerlMonks: Click on the "Read more" link below to see the rest of the article)
Let's start with the config file for the server (server.xml):
<clacks> <appname>Clacks Server</appname> <!-- This is the magic that makes Unix domain sockets possible In most (is not all) cases, you should use absolute paths. The only reason this one isn't: I don't know anything about *your* system, so i used only the filename. If you are so inclined, you can specify multiple <socket> lin +es to listen on multiple unix domain sockets... ...for whatever reason you might have --> <socket>example.sock</socket> <!-- If you want TCP, need some valid openssl certs. Then you can +do something like this: <ip>127.0.0.1</ip> <port>49888</port> <ssl> <cert>exampleserver.crt</cert> <key>exampleserver.key</key> </ssl> Optionally, you can then also comment out the <socket> option + you don't want to (or can't) use Unix Domain Sockets --> <pingtimeout>600</pingtimeout> <!-- ping timeout when running clacks servers in a master/slave co +nfig --> <interclackspingtimeout>60</interclackspingtimeout> <!-- Make the in-memory key/value store persist across restarts -- +> <persistancefile>clackspersistance.dat</persistancefile> <!-- This is the main user that has all permissions and also Inter +clacks --> <username>exampleuser</username> <password>unsafepassword</password> <!-- additional user account with read and write, but no "manage" +permissions --> <user> <username>rwuser</username> <password>foo</password> <read>1</read> <write>1</write> <manage>0</manage> </user> <!-- read-only user. Can see everything, but can't change it --> <user> <username>rouser</username> <password>bar</password> <read>1</read> <write>0</write> <manage>0</manage> </user> <!-- some experimental settings to reduce CPU usage on low-traffic + clacks servers --> <throttle> <maxsleep>100</maxsleep> <step>5</step> </throttle> </clacks>
The server code itself is simple, we basically just have to instantiate the Net::Clacks::Server object with a path to the XML config file and call run().
It's as easy as that. Here's the perl code (server.pl):
#!/usr/bin/env perl #---AUTOPRAGMASTART--- use v5.36; use strict; use diagnostics; use English qw(-no_match_vars); use Carp qw[carp croak confess cluck longmess shortmess]; our $VERSION = 29; use autodie qw( close ); use Array::Contains; use utf8; #---AUTOPRAGMAEND--- my $isDebugging = 0; BEGIN { if(defined($ARGV[1]) && $ARGV[1] eq "--debug") { print("Development INC activated\n\n"); unshift @INC, "../lib"; # Point to your local copy of Net-Clac +ks/lib if you want to play around with the server code without "make +install" every time you change something $isDebugging = 1; } }; use Net::Clacks::Server; my $configfile = shift @ARGV; croak("No Config file parameter") if(!defined($configfile) || $configf +ile eq ''); my $worker = Net::Clacks::Server->new($isDebugging, $configfile); $worker->run;
The first startup of the server should look something like this:
$ perl server.pl server.xml --debug Development INC activated PC_CONFIG_PATHS undefined, falling back to legacy mode Loading config file server.xml ------- Parsing config file server.xml ------ Removing old unix domain socket file example.sock Listening on Unix domain socket example.sock Sorry, no valid persistance file found. Starting server 'blankety-blan +k' Ready. Saving persistance file
You can ignore the warning about PC_CONFIG_PATHS for this tutorial. Basically, it says that you can set the environment variable PC_CONFIG_PATHS to the folder in which it would look for the XML config file. This setting is mostly there so you can point the server to different configuration directories without having to change the startup call. (I use this in my PageCamel framework so i can quickly load different projects without having to change start scripts or move directories around).
The first thing we want to do is write a very simple client that can just MONITOR all activity on the server, which is quite handy if we run into problems and want to debug communications. We will use the NOPING and MONITOR commands to make the code as simple as possible.
#!/usr/bin/env perl #---AUTOPRAGMASTART--- use v5.36; use strict; use diagnostics; use English qw(-no_match_vars); use Carp qw[carp croak confess cluck longmess shortmess]; our $VERSION = 29; use autodie qw( close ); use Array::Contains; use utf8; use Encode qw(is_utf8 encode_utf8 decode_utf8); use Data::Dumper; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Time::HiRes qw(sleep); my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'debugmonitor'; my $keepRunning = 1; $SIG{TERM} = sub{ print "SIGTERM, exiting program\n"; $keepRunning = 0; }; # Client side caching is a whole other topic. Basically, it doesn't re +send stuff when the value hasn't changed # In most cases this isn't required or can even be counter-productive. + This is one of those advanced topics... my $is_caching = 0; # Let's connect my $debug = Net::Clacks::Client->newSocket('example.sock', $username, +$password, $applicationname, $is_caching) or croak("Connection to server failed"); # If you want to use TCP, you can do this instead: #my $debug = Net::Clacks::Client->new('127.0.0.1', 49888, $username, $ +password, $applicationname, $is_caching); $debug->setMonitormode(1); # NSA mode activate ;-) $debug->disablePing(); # Disable ping timeouts $debug->doNetwork(); # Send buffered commands to server while($keepRunning) { $debug->doNetwork(); # Sync with server # Iterate over all messages in our in-buffer while((my $msg = $debug->getNext())) { if($msg->{type} eq 'debug') { print $msg->{host}, ": ", $msg->{command}, "\n"; } elsif($msg->{type} eq 'serverinfo') { print '+++ Connected to server version: ', $msg->{data}, " +\n"; } elsif($msg->{type} eq 'disconnect') { print '+++ Disconnected by server, reason given: ', $msg-> +{data}, "\n"; last; } } sleep(0.2); } $debug->disconnect(); exit(0);
Let's write a simple worker that reads a (fake) sensors cyclicly and also counts the number of sensor readings it has done. It will also get it's cycle time from the key/value store in clacks, defaulting to 5 seconds if none is set. (fakesensor.pl):
#!/usr/bin/env perl #---AUTOPRAGMASTART--- use v5.36; use strict; use diagnostics; use English qw(-no_match_vars); use Carp qw[carp croak confess cluck longmess shortmess]; our $VERSION = 29; use autodie qw( close ); use Array::Contains; use utf8; use Encode qw(is_utf8 encode_utf8 decode_utf8); use Data::Dumper; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Time::HiRes qw(sleep); my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'debugmonitor'; my $keepRunning = 1; $SIG{TERM} = sub{ print "SIGTERM, exiting program\n"; $keepRunning = 0; }; my $cycletime = 5; my $nextping = time + 10; my $nextrun = 0; # Client side caching is a whole other topic. Basically, it doesn't re +send stuff when the value hasn't changed # In most cases this isn't required or can even be counter-productive. + This is one of those advanced topics... my $is_caching = 0; # Let's connect my $clacks = Net::Clacks::Client->newSocket('example.sock', $username, + $password, $applicationname, $is_caching) or croak("Connection to server failed"); initClacks(); while($keepRunning) { $clacks->doNetwork(); # Sync with server if($nextping < time) { $clacks->ping(); # Send a keepalive ping $nextping = time + 10; } # Iterate over all messages in our in-buffer while((my $msg = $clacks->getNext())) { if($msg->{type} eq 'disconnect') { print '+++ Disconnected by server, reason given: ', $msg-> +{data}, "\n"; sleep(2); # Wait a couple of seconds, should automatically + reconnect on next doNetwork initClacks(); # Need to resend listen() and stuff } elsif($msg->{type} eq 'serverinfo') { print '+++ Connected to server version: ', $msg->{data}, " +\n"; } elsif($msg->{type} eq 'set' && $msg->{name} eq 'FAKESENSOR:: +cycletime') { $cycletime = 0 + $msg->{data}; print "Cycle time has changed to $cycletime seconds\n"; } } if($nextrun < time) { # Do a cycle $nextrun = time + $cycletime; my $newval = int(rand(1000)); print "New sensor value: $newval\n"; $clacks->setAndStore('FAKESENSOR::value', $newval); # broadcas +t new value and also store it $clacks->increment('FAKESENSOR::count', 1); # Increment the se +nsor reading count by 1 (stepsize is an optional argument). Autovivif +ies the value if not defined $clacks->doNetwork(); } sleep(0.2); } $clacks->disconnect(); exit(0); sub initClacks() { # We use the key/value store for long-term storage... my $ct = $clacks->retrieve('FAKESENSOR::cycletime'); if(!defined($ct)) { # Fallback to default of 5 seconds print "Storing default cycletime in clacks\n"; $clacks->store('FAKESENSOR::cycletime', $cycletime); } else { $cycletime = 0 + $ct; print "Got a preset cycletime of $cycletime seconds\n"; } # ...but LISTEN to real-time changes. This implies the client chan +ging this value will use setAndStore() $clacks->listen('FAKESENSOR::cycletime'); $clacks->doNetwork(); return; }
Ok, now we got a server and a fake sensor, let's make another client that reacts to sensor readings in real time. We can mostly recycle fakesensor.pl to do this (tracker.pl)
#!/usr/bin/env perl #---AUTOPRAGMASTART--- use v5.36; use strict; use diagnostics; use English qw(-no_match_vars); use Carp qw[carp croak confess cluck longmess shortmess]; our $VERSION = 29; use autodie qw( close ); use Array::Contains; use utf8; use Encode qw(is_utf8 encode_utf8 decode_utf8); use Data::Dumper; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Time::HiRes qw(sleep); my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'debugmonitor'; my $keepRunning = 1; $SIG{TERM} = sub{ print "SIGTERM, exiting program\n"; $keepRunning = 0; }; my $nextping = time + 10; my $nextrun = 0; # Client side caching is a whole other topic. Basically, it doesn't re +send stuff when the value hasn't changed # In most cases this isn't required or can even be counter-productive. + This is one of those advanced topics... my $is_caching = 0; # Let's connect my $clacks = Net::Clacks::Client->newSocket('example.sock', $username, + $password, $applicationname, $is_caching) or croak("Connection to server failed"); initClacks(); while($keepRunning) { $clacks->doNetwork(); # Sync with server if($nextping < time) { $clacks->ping(); # Send a keepalive ping $nextping = time + 10; } # Iterate over all messages in our in-buffer while((my $msg = $clacks->getNext())) { if($msg->{type} eq 'disconnect') { print '+++ Disconnected by server, reason given: ', $msg-> +{data}, "\n"; sleep(2); # Wait a couple of seconds, should automatically + reconnect on next doNetwork initClacks(); # Need to resend listen() and stuff } elsif($msg->{type} eq 'serverinfo') { print '+++ Connected to server version: ', $msg->{data}, " +\n"; } elsif($msg->{type} eq 'set' && $msg->{name} eq 'FAKESENSOR:: +value') { print "Fake sensor reports a value of: ", $msg->{data}, "\ +n"; } } sleep(0.2); } $clacks->disconnect(); exit(0); sub initClacks() { # LISTEN to real-time changes. This implies the client changing th +is value will use setAndStore() $clacks->listen('FAKESENSOR::value'); $clacks->doNetwork(); return; }
Making a script that reads the last known value just one time is even easier. We don't need to bother listening, looping and all that jazz (readsensor.pl):
#!/usr/bin/env perl #---AUTOPRAGMASTART--- use v5.36; use strict; use diagnostics; use English qw(-no_match_vars); use Carp qw[carp croak confess cluck longmess shortmess]; our $VERSION = 29; use autodie qw( close ); use Array::Contains; use utf8; use Encode qw(is_utf8 encode_utf8 decode_utf8); use Data::Dumper; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Time::HiRes qw(sleep); my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'debugmonitor'; my $keepRunning = 1; $SIG{TERM} = sub{ print "SIGTERM, exiting program\n"; $keepRunning = 0; }; # Client side caching is a whole other topic. Basically, it doesn't re +send stuff when the value hasn't changed # In most cases this isn't required or can even be counter-productive. + This is one of those advanced topics... my $is_caching = 0; # Let's connect my $clacks = Net::Clacks::Client->newSocket('example.sock', $username, + $password, $applicationname, $is_caching) or croak("Connection to server failed"); $clacks->doNetwork(); # Sync with server # RETRIEVE stored value from clacks server my $val = $clacks->retrieve('FAKESENSOR::value'); if(!defined($val)) { print "No known sensor data\n"; } else { print "Last sensor value: $val\n"; } $clacks->disconnect();
Of course, it's just as easy to do a one-time write to change the fakesensor cycle time (setcycletime.pl):
#!/usr/bin/env perl #---AUTOPRAGMASTART--- use v5.36; use strict; use diagnostics; use English qw(-no_match_vars); use Carp qw[carp croak confess cluck longmess shortmess]; our $VERSION = 29; use autodie qw( close ); use Array::Contains; use utf8; use Encode qw(is_utf8 encode_utf8 decode_utf8); use Data::Dumper; #---AUTOPRAGMAEND--- use Net::Clacks::Client; use Time::HiRes qw(sleep); my $username = 'exampleuser'; my $password = 'unsafepassword'; my $applicationname = 'debugmonitor'; my $keepRunning = 1; $SIG{TERM} = sub{ print "SIGTERM, exiting program\n"; $keepRunning = 0; }; my $cycletime = shift @ARGV; if(!defined($cycletime)) { print "Usage: perl cycletime.pl CYCLEINSECONDS\n"; exit(0); } $cycletime = 0 + $cycletime; # Make sure its a number if(!$cycletime || $cycletime < 0) { print "Cycletime must be a positive numer and can not be zero\n"; exit(0); } # Client side caching is a whole other topic. Basically, it doesn't re +send stuff when the value hasn't changed # In most cases this isn't required or can even be counter-productive. + This is one of those advanced topics... my $is_caching = 0; # Let's connect my $clacks = Net::Clacks::Client->newSocket('example.sock', $username, + $password, $applicationname, $is_caching) or croak("Connection to server failed"); # SETANDSTORE new cycletime. This sends a real-time message AND stores + the value in the key/vallue store for later retrieval $clacks->setAndStore('FAKESENSOR::cycletime', $cycletime); $clacks->doNetwork(); # Sync with server $clacks->disconnect();
I hope you found this mini-tutorial somewhat useful. The examples i provided just implement a very tiny landscape of messaging and data. For such a simple project, there are most likely better solutions.
But in my projects, it's not unusual to have hundreds of sending and listening endpoints, with dozens of modules using Net::Clacks to centrally store variables and/or cache data. Some of these across multiple servers, where the interclacks subsystem really comes to shine in reducing traffic between those servers (each clients only connects locally via Unix domain sockets).
In reply to Net::Clacks for IPC (2024 tutorial) by cavac
For: | Use: | ||
& | & | ||
< | < | ||
> | > | ||
[ | [ | ||
] | ] |