michellem has asked for the wisdom of the Perl Monks concerning the following question:

Hello fellow monks, I'm looking for a bit of advice on the relative merits of running a script via cron on some interval, or having a process running. This is the basic scenario:

I have a web-database/CMS system, where, right now 100% of the work is done by cgi's acting on a database. There are some functions I'd like to have happen independent of the web server, such as specific actions taken at specific time intervals (like, for example, sending an email when a record is about to expire, etc.). It seems in order to have these sorts of things happen, I have three options:

1) Each time a script is called by the web server, it checks to see whether or not any actions are pending, then does them.
2) I set up a script to be run by cron on specific intervals, that checks for pending actions, and completes them.
3) I have a running script that periodically checks for pending action

An important thing to know is that I can't imagine that people will have actions that will happen with any more frequency than once an hour. Option 1 is my least favorite, because it depends on use of the web cgi's - which is not at all predictable. I'm leaning toward using cron, but I can imagine some of the advantages of having a running process.

Ideas, or experiences? Thanks!

Replies are listed 'Best First'.
Re: Cron or running process?
by freddo411 (Chaplain) on Apr 28, 2004 at 20:49 UTC
    Use cron. Cron essentially acts like a running process that fires off things when necessary, (yes, I know that this isn't strictly true...) why reinvent the wheel?

    That being said, cron does have a few gotchas:
    * Funny things happen when DST kicks in or out (you can search perlmonks for my recent thread on this.)
    * Scripts run from cron have a pristine environment, without a normal path env var etc, etc. You must keep this in mind when coding your scripts.

    Editing a crontab file to schedule your scripts is not at all difficult. Consult man crontab for details.

    Nothing is too wonderful to be true
    -- Michael Faraday

Re: Cron or running process?
by Elijah (Hermit) on Apr 28, 2004 at 20:44 UTC
    Well in your specific application it appears that option 2 and 3 are very similar (with regards to frequency of execution) so the added system resources being chewed up by a deamon running constantly in the background does not outweight the need to be real time. I would go with number 2. This option for your scenerio will be most practical, applicable and resource friendly.

    If you fell the need to be almost real time you can setup a cron job to run every minute and check the target files and still be more efficient than a running app in the background. Even if a running app does this very thing (meaning run at startup and poll the file every 60 seconds then sleep for 60 seconds and do it again) it is still using system resources while sleeping.
      I'd go for cron, keeping in mind that:
      • It could hang and not work (though the same could be for your own code, and I guess that would be more likely than cron itself)
      • More than one instance of your script could be running at the same time. Play it safe and check before doing things that will break if done by two different process at once. Having a database backend with transactions might help.
      • Log, log and log again. You want to know whether your thing ran, whether it completed okay, what went wrong. Log it before, don't wonder what went wrong afterwards.
      I guess this is my $0.02
Re: Cron or running process?
by haoess (Curate) on Apr 28, 2004 at 20:51 UTC
    Option 3 is the reason why cron was invented. Option 3 is much more work: Your process can be stopped, killed, it can hang etc. Every the machine reboots, you have to restart your process by hand. Use cron.
      Not that I am an advocate for running process as my first post has indicated but restarting the app by hand is not neccessary if you add an entry to "/etc/rc.d/rc.local" (For RH, Fedora) and a similar file path for other distros. This entry will start the script automagically when the OS boots.
Re: Cron or running process?
by Ryszard (Priest) on Apr 29, 2004 at 08:36 UTC
    As others have said 2 & 3 are similar.

    I've done it both ways for various things, personally I like the ease of cron, but it has some disadvantages you should know as well:

    1. Cron can crash, if you're not aware of this, then your entire process will fail.
    2. cron can be tricky to work with as it doesnt source and environment, stuff that works interactively may not work with cron.

    OTOH, if you're daemonising the process you can do some cool stuff like starting it via the init.d, you can see if it is explicitly alive by looking at the process table. The down side is it may die at any time for any reason, memory, a bug, whatever.

    Bottom line, is you do what you're comfortable with, and add monitoring. You want to know if your process is alive, or your cron job has run, you want to be alerted if your job abends, or your daemon fails.

    There are a couple of methods i use which seem to work quite well.. checking the last update of your log file, will implicitly tell you everything is ok. If possible you should monitor the functionality of your daemon/job, so you can explicitly tell if its run..

      ++ to the parent message.

      To this point, cron's default behavior is to email all output to the user whose crontab invoked the process. Often times, the crontab output is redirected somewhere else , /dev/null being a popular choice, or other useful log file.

      I tend to have my cron processes email success emails to one address, and failure to another address.

      Good sysadmin's hook their processes into their favorite monitoring tools, TMTOWTDI.


      Nothing is too wonderful to be true
      -- Michael Faraday

Re: Cron or running process?
by shonorio (Hermit) on Apr 29, 2004 at 00:08 UTC
    On my opinion, keep your system simples as possible, and to have a good system on 'running process' you will must write code unnecessary.

    Solli Moreira Honorio
    Sao Paulo - Brazil
Re: Cron or running process?
by z3d (Scribe) on Apr 29, 2004 at 08:36 UTC
    If you run it from cron, just be sure to have "safety catches" (only way I can think to say it). I've faced this problem in the past also, went with cron, which works fine accept on that one in a thousand runs chance that the cron job hangs for some reason, which results over time in multiple copies of the same job running...bah, you know what I mean I hope :) Especially since you mentioned DB and email connections, which means your relying on returns from them to be timely.

    "I have never written bad code. There are merely unanticipated features."
Re: Cron or running process?
by weierophinney (Pilgrim) on Apr 29, 2004 at 17:27 UTC
    I've done all three options you present. I disliked (1) because (a) you can't necessarily count on hits to your website occurring, (b) your web server process could fail, which means it wouldn't be run, and (c) it adds extra overhead to your CGI process, thus slowing it down for the user.

    I use (2), cron, frequently, and use it for many things. However, I've also had cron fail on me (and I haven't been able to get it back on the system where it failed, further compounding the issue), as well as have new cron-spawned processes collide with previous cron-spawned processes. Regardless, cron is my favorite option of the three you present, especially when you need worry about neither flexibility of schedule nor collisions.

    I use (3) currently for a process that monitors servers. It's more flexible than cron, but it takes more resources (for instance, it needs to spawn the perl engine).

    If you want to go the route of (3), your own daemon, it's actually very simple to do in perl, and not terribly resource intensive, if you use the Proc::Daemon module. It's as simple as:

    use Proc::Daemon; Proc::Daemon::init; while (1) { # do some code sleep 60; # wake up every minute }
    If you don't want to worry about the process dying/crashing and not restarting, add a line to your /etc/inittab:
    where "MD" is a unique identifier for your daemon, and 2345 indicate which runlevels to start your script on.
Re: Cron or running process?
by BuddhaNature (Beadle) on Apr 30, 2004 at 17:05 UTC
    Well I am not sure it is 100% applicable to your situation, but you might want to check out Monit. It is what I like to call a gigantic if/then statement. Basically it is a program that will watch just about anything (a process, a file, a directory, a port, etc.) and will do something if anything changes about said watched "thing."

    For instance, you have a daemon running (say sshd) and you want to make sure it keeps running - monit will periodically check that it is still running. If it goes down it can mail you and/or restart the daemon for you, or do just about anything else you want it to.

    Maybe you need to keep an eye on a config file (like httpd.conf) so that if it changes you can restart a daemon automatically to make the changes live (like apache) - monit will do this for you.

    Sorry if this sounds like a plug, but it is a nice program. Hope this helps.