llancet has asked for the wisdom of the Perl Monks concerning the following question:

I want to have:
1: several daemon processes, each one holds a workflow.
2: a GUI process, which shows those workflows' status (which step is running), and controls the workflows (start,stop, add/remove flow step).

So, at least, daemons should be able to send their status to GUI, and can receive control commands from GUI. Which kind of IPC mechanism should I use? TCP? Named pipe?

Another question: there should be some mainloop inside each daemon. Which one is better? POE? Glib?

  • Comment on How to implement such kind of GUI - daemon cooperation?

Replies are listed 'Best First'.
Re: How to implement such kind of GUI - daemon cooperation?
by Corion (Patriarch) on Oct 12, 2010 at 07:25 UTC

    I would base the choice of IPC mechanism on what other requirements there are:

    • Should the workflow elements be able to run on different machines than the GUI? If so, I would look away from named pipes or unix domain sockets, and look at a network based messaging mechanism like TCP, UDP, syslog, or even SMTP.
    • Does the GUI have file system access to all directories where the workflow elements run? If so, a simple log file for each element might be enough. Reconfiguration etc. would then require signals, if the OS provides an implementation.
    • How critical is reconfiguration over transactional stability? If reconfiguration is less important than finishing a transaction, you can get away with having the main loop checking for a configuration change and alternatively processing the next item. That way, you will always process an item completely using a single configuration. Also look into versioned workflows as Workflow discusses them.

    As for the choice of mainloop, I would really try to avoid introducing POE or Glib or any other complex mechanism into a workflow element. Try to keep these as small, self-contained external programs. This makes them easily testable and also forces you to define clear boundaries and clear input and output specifications. Having multiple external processes also makes it very easy to scale up your process over more machines or CPUs.

      Currently, I'm not thinking about distribute the GUI and daemons on different machine. So I should use named pipe? Or just using process signals and a file?

      In my design, those daemons are not able to be re-configured unless they are not at "running" state. However, I think it still needs to be sychronized that configure files are written down before corresponding daemon starts.

      With out a mainloop, how can a flow daemon kept itself exist? Just by while (1) {}? However it seems while(1){} is very CPU-consuming. I don't know why...

Re: How to implement such kind of GUI - daemon cooperation?
by locked_user sundialsvc4 (Abbot) on Oct 12, 2010 at 13:52 UTC

    As Corion has suggested, you have many design choices to make here, and no one here can realistically make these choices for you.

    In the experience of many, a “loosely coupled” (so to speak...) system, linked by named-pipes, is often the most satisfactory arrangement because it can be scaled-up among multiple computer systems fairly easily.   Each of the worker daemons reads requests from a pipe, and posts completion status (including in-progress status reports) to another pipe or pipes.   Each daemon is also responsible for reading its own configuration files, and is obliged to lock and unlock those files (or to choose an appropriately exclusive open-mode) to ensure their integrity while doing so.

    The daemons are running under the overall control of some kind of a workload-manager, which doles out work and handles any status messages.   It keeps its own “tote board” of what is going on.   Probably the easiest way to handle a monitoring GUI, then, is to define another type of request, which the GUI can make.   This request is handled by the monitor process in exactly the same way as any other.   The GUI fires requests using a timer, then updates its display based on the response received.   (If the timer goes off again before the first response is returned, the GUI does not send another one.)

    What you’ll notice about a design like this is that ... it’s asynchronous.   The pipes are acting as rubber-hoses between the various pieces, each of which can basically be single-threaded.   It does not presuppose the existence of a network file-system.

    What you’ll also notice about a design like this is that, it is extremely common.   The world is full of systems just like this.   They are a dime a dozen.   Yes, there are plenty of CPAN modules for building them.   And, no, I’m not going to suggest one.