QuillMeantTen has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, I have been working on some code for a daemon. It's role will be to provide messenger daemons to other perl processes so they can communicate through a standard interface whether they be on separate hosts or on the same machine. here are the first strokes of code : 

sub run {
    my $request_file_name = shift;#named pipe to which the Messenging 
+daemon will
                       #listen for messenger spawn requests.
    if (!defined($request_file_name)){
        croak "can not run without something to listen to!\n";
    }
    mkfifo($request_file_name,0777)||croak "could not open named pipe 
+: $!";
    my %children = ();
    my $pid;
    open my $fh,'<',$request_file_name;
    while(1){
        my $params = fd_retrieve($fh);
        my $messenger = $class->new($params);
        $pid = fork();
        if (!$pid){
            $children{$pid} = 1;
        }
        else{
            $messenger->relay;
        }
    }
}

[download]

the $class variable is a package variable that contains the name of the Messenger class (the little daemons created by the big one). other scripts will request a new messenger by nstoring a param hash to the named pipe, said hash containing filehandles for input and output as well as all needed informations for the creation of a new daemon.

the idea behind it is to allow the following kind of interaction : 1)Reporting daemon receive data from database script (from a messenger between him and the database script)

2)he decides to create a report object and send it to the Monitoring daemon which resides on another machine

3)Reporting daemon asks his local head of messengers daemon for a messenger to communicate with his boss on remote machine(he has to create himself the sockets and such, that's the first thing that puts me ill at ease, I'm not sure if it wouldn't be better for the Messenger daemon to do that kind of thing).

4)Messenger daemon creates a lesser messenger (the little guy only role is to replay what he gets from one input on n>0 outputs) and asks the remote Messenger daemon to do a symetrical setup

5)remote Messenger daemon prods Monitoring daemon with the news : here is some data for you to handle

up to now I thought I would do most of it with named pipes but what I recently read about unix sockets makes me hesitate and that hesitation plus my second thoughts on the whole design are making me ask for guidance here before I launch myself in some quixotic endeavor. Thanks for putting up with me this far :)

Replies are listed 'Best First'.
Re: IPC Design problem
by kcott (Archbishop) on Aug 25, 2015 at 03:33 UTC

    G'day QuillMeantTen,

    You wrote:

    "... whether they be on separate hosts or on the same machine."

    The first line of "perlipc: Named Pipes" has (my emphasis):

    "A named pipe (often referred to as a FIFO) is an old Unix IPC mechanism for processes communicating on the same machine."

    So, given you want the ability to communicate across hosts, named pipes would not be a suitable solution.

    Sockets would be a better option. There's lots of documentation and sample code in perlipc.

    — Ken

[reply]

      I know that I had the idea to chain messengers.

      Say gate ones that would use sockets and interior one using named pipes or Unix sockets ( I could implement both and choose depending on the platform).

      What I'm not so sure about is the design choice of letting anyone but a head of messengers demon open the necessary file handles :-$



      Update I'm going to give it a try that way : only the head of messengers do the file handle management, he shall use a named pipe for his orders (thus making itself easily cross platform) and shall receive his working parameters as a hash of the following type : 

      my %params = (
    input=>{name=>'filename',
            type=>'unix_socket|network_socket|named_pipe',
            bind=>address,#if network socket},
    output=>{output1=>'filename|socket_name',
             output2=>'idem',
             ...
             ...
             },
);

      [download]


      then he shall open the file handles and create a child process with a little messenger demons that will relay data to and from them. It seemed more elegant than the first idea where I count on other modules to make the right parameter hash for the messenger demon instanciation method (furthermore it should allow me to update this method without having to change code everywhere).

      I will post the resulting code in the CUFP section, might give some of you quite a laugh (I'm quite new to multi process programing).
      Cheers!

[reply]
[d/l]
Re: IPC Design problem
by anonymized user 468275 (Curate) on Aug 25, 2015 at 10:52 UTC
    It's not obvious what happens when a child hits the end of the infinite while loop, but I would expect it will keep looping and become a parent of its own children. It's usually correct for child processes to explicitly exit when they are done DWIMming.

    One world, one people

[reply]

      So I think I made some headway. I have most of the main messaging daemon code down but I'm having a bit of a problem with nstore...

      here is Messaging.pm :

      Read more... (9 kB)

      here is Messenger (the little daemon class)



      Read more... (4 kB)

      and finaly the code I'm using to test them. I first run messaging.pm with one parameter : the filename for the named pipe

      then I run the following script :

      Read more... (918 Bytes)

      has soon as it is done writing to the pipe and has closed it I receive the following error message from my messaging daemon :

      Magic number checking on storable file failed at /usr/local/lib/perl/5.14.2/Storable.pm line 401, <$fh> chunk 1, at ../lib/Security/Monitoring/Messaging.pm line 115

      I find it most disturbing because it means that my previous tests on the Messaging.pm file must have been cheating someway, if they had not they would have created the same kind of error yet they run fine! ->those tests, I mean

      Read more... (3 kB)


      If you have an inkling of what's wrong, even an intuition I would be most grateful because I'm quite lost...

      Also in the test file I used a shotgun to scatter some kill commands everywhere in the subroutine because if I did not the test output will be out of sequence and appear three times.
      I tried to find out how I could prevent my children from doing that whatever the scheduler decided but I could not, so if you have any better/more elegant way to do it I'm all ears!

      Thank you for reading :)

[reply]
[d/l]
[select]

        I think your read loop in the Messaging class is a little too greedy:

        ABORT:while(<$fh>){
        my $data = $_;
        my $params = fd_retrieve($fh);
        ...

        [download]

        This first reads a line, then does an fd_retrieve(). Your test script first does an nstore_fd(), then writes some text, i.e. exactly the other way around.

        This is not the only problem with your code. Apart from style/idiom issues, the fact that you are mixing plain text with binary text in the same channel is asking for trouble. If you reverse the writing order in your test script, the fact that the text string in our test script does not contain a newline will most likely trip up the <$fh> statement and slurp up the nstore-d data as well. (And if you have a string with more than one newline you also have a problem.) You should really consider using some appropriate encapsulation, like a hash with both the data and the parameters, written out with nstore: 

          # Writing a message:
  my $message = {
    data => "Some message\nthat can contain newlines\n",
    params => \%params,
  };
  nstore_fd($message, $fh);

  # Reading a message:
  my $message = fd_retrieve($fh);
  my $params = $message->{'params'};
  my $data = $message->{'data'};

        [download]

        ... Or something like that.

        Hope this helps!

        -- Steven

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom