Dallaylaen has asked for the wisdom of the Perl Monks concerning the following question:

Hello dear esteemed monks,

We need to daemonize a queue-processing program (been using uWSGI for that, but it's kind of awkward). So I'm looking for a module that would do the trick for me. Specifically I need the following:

1. forking a configurable number of worker processes;

2. Starting/stopping them (via a pid file?);

3. Graceful restart - the old workers can keep processing their last request for as long as needed, and the new ones are spawned to take over;

4. Ability to specify number of requests being processed after which a child exits and is replaced by a new one.

While I'm definitely able to write all of the above, maybe there is a module that already does that. Any::Daemon seems close to the above spec but 10% test coverage is a bit scary. Any suggestions?

UPDATE: We're running Linux. Although I assume a "generic daemonizing" module should run under other UNIX systems as well.

Thank you.

Replies are listed 'Best First'.
Re: Looking for a daemonizing module
by rjt (Curate) on Sep 07, 2019 at 08:11 UTC

    Any::Daemon seems alright, and Parallel::ForkManager might also be a reasonable choice for fork-model multiprocessing apps in Perl, even though it doesn't do a few of the things on your list.

    From the synopsis:

    use Parallel::ForkManager;
 
$pm = Parallel::ForkManager->new($MAX_PROCESSES);
 
foreach $data (@all_data) {
  # Forks and returns the pid for the child:
  my $pid = $pm->start and next;
 
  ... do some work with $data in the child process ...
 
  $pm->finish; # Terminates the child process
}

    [download]

    I'm not sure how easy it will be to do an Apache-style graceful restart (your #3), as I haven't used P:FM in a while.

    As for #4, that would be trivial to implement in the child itself. $pm->finish if ++$my_requests > $conf->{max_child_requests};

    It doesn't give you any kind of PID file or command line help (Daemon::Control is nice for that, but it isn't really intended for multiprocessing), which is way harder to get right than a PID file.

    I hope this helps. Without knowing more about your application, this advice is generic. Specific requirements might pull you in a different direction.

    use strict; use warnings; omitted for brevity.
[reply]
[d/l]
[select]

      Parallel::ForkManager seems to do what I want. I knew about it, but somehow overlooked it in my search this time. Thank you.

[reply]
Re: Looking for a daemonizing module
by karlgoethebier (Abbot) on Sep 07, 2019 at 16:28 UTC
[reply]
[d/l]
      A great collection, thank you!
[reply]
Re: Looking for a daemonizing module
by stevieb (Canon) on Sep 06, 2019 at 21:43 UTC

    Please advise on what Operating System your application will be running under. I assume some form of Unix, but it'd be best if you'd edit your Original Post and state it inherently.

[reply]
Re: Looking for a daemonizing module
by shadowsong (Pilgrim) on Sep 07, 2019 at 23:48 UTC
[reply]
Re: Looking for a daemonizing module
by 1nickt (Canon) on Sep 09, 2019 at 06:42 UTC

    Hi, I would not advise daemonizing your program inside itself, nor to use Perl for it. I suggest writing your program and making it capable of running in single-process mode or with multiple workers (using MCE you can achieve single-process mode via the number of workers flag), and then separately installing and configuring it as a service (daemon) on the machines where it will run. The OS and standard utilities are much better designed to handle this and all related tasks, such as log rotation, startup at boot time, etc.

    FWIW I recommend daemontools for Unices.

    Hope this helps!


    The way forward always starts with a minimal test.
[reply]
      I recommend daemontools for Unices.

      I can only second that. Writing daemons to be compatible with daemontools (i.e. forget about almost all aspects of running as a daemon) automatically makes them compatible with tons of other tools that copied the daemontools idea (almost all alternatives to a classic init, xinetd, and even the nightmare systemd).

      See also:

      Just another little note: If you write your daemon to be compatible with daemontools, it can be incredible compact. A hello world deamon can be written as a bourne shell script in less than 10 lines of code:

      #!/bin/sh
# blabbyd.sh: a blabby daemon, version 2
#
# note: variables now provided from environment with envdir:
#
#   .env/MESSAGE
#   .env/PAUSE
#
echo "starting blabbyd..."
# loop forever
while :
do
    echo "blabbyd: ${MESSAGE}"
    echo "blabbyd: sleeping for ${PAUSE} seconds..."
    sleep ${PAUSE}
done
# that's all, folks!

      [download]

      (Copied right out of the djb way)

      All that you think is missing for a daemon is already implemented by daemontools or some other, compatible software. You don't have to reinvent the wheel.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]

Back to Seekers of Perl Wisdom