traveler has asked for the wisdom of the Perl Monks concerning the following question:

I'd like to be able to have a function (that takes about one second) to run every minute. It samples low frequency system data and generates a plot. I'd really like one point per minute. Any thoughts would be helpful. Here's what I've tried and the problems I've had:
  1. Used sleep 60 at the end of the function. This gives a roughly 61 second period for executing the function. (Because the function takes about a second to execute.) That means that data for some minutes will be skipped.
  2. Used alarm 60 in a signal handler for $SIG{ALRM}. The clock seems to creep with this, too, but I am not sure why. It looks as though I get an interrupt every 60.5 seconds. I'll still lose data points this way.
  3. Used
    select undef, undef, undef, 1.0; run_func if ! (time%60);
    This seemed to skip every other minute for reasons I cannot figure out.
In cases 1 and 2 above, I've sat in a wait loop (wait while (1);) as the body of the program.

I'm using perl 5.005_003 on Linux 2.2.16 (mostly Red Hat). If it makes any difference, the function (run_func above) is a read of about 50 lines using XML::Simple, a few additions, and a plot made via GD::Graph.

--traveler

Replies are listed 'Best First'.
Re: Reasonably accurate timing
by bikeNomad (Priest) on Jul 23, 2001 at 20:35 UTC
    I'm doing data sampling on several channels, and found that Event was the best way to go. You can have timed watchers, and mix them if you need with watchers that wait for IO, or do stuff in the background.

    Use the hard setting to make sure that the period is constant.

    It can use Time:HiRes if you have it.

      Thanks. This seems to mostly work (useful module). I did set hard to 1, but I am still seeing timer events "creep" forward at about .5 seconds/minute. Any ideas?

      Update: The documentation says that the interval attribute is added to the time-to-execute (called at) after the callback returns. That means that the interval really is interval+time_taken_by_callback. So, the question really is, how do I make the interval just interval?

        That's what hard is supposed to be for (not adding interval after, but before). Make sure you include Time::HiRes. From the manpage:
        If the "hard" flag is set, the event will be queued for execution relative to the last time the callback was invoked. However, if "hard" is false the new timeout will be calculated relative to the current time (this is the default).

        If you set repeat and hard, it shouldn't creep. I start up my samplers parked, set the at and interval, and then start them.

        How fast are you getting callbacks? If you are taking too long in some of the callbacks, you could bump the callback time. Make sure that your callback action can never overlap the next callback by either speeding up the callback or slowing down the rate. Also, you might try setting debugging on either globally or on the watcher; there are diagnostics printed as to the actual times between callbacks, etc.

Re: Reasonably accurate timing
by lhoward (Vicar) on Jul 23, 2001 at 21:55 UTC
    How about sleeping until the next polling interval (based on when you started looping) instead of a hardcoded delay? This will account for however long your processing takes (even if the amount of time it takes to process changes). It can be "off" by a fraction of a second, but should correct itself so the drift does not accumulate.
    my $t0=time(); while(1){ # do processing # .. # sleep till next polling interval. sleep 60-(time()-$t0)%60; }
    Use this concept with Time::HiRes for greater accuracy.
Re: Reasonably accurate timing
by grinder (Bishop) on Jul 23, 2001 at 22:24 UTC
    Why don't you just set it up as a cron job? Right now, 99.16% of time it's just tying up resources. This also reduces the chance of the script being blown away by accident. Next minute, it'll be back again...

    --
    g r i n d e r
      Normally that would be a good solution. Unfortunately, this script does the following:
      1. Read in a (possibly) very large file and process it.
      2. Process and plot data every minute.
      The data from phase 1 is needed in phase 2. I actually plan on running the script from init...
        I recommend mixing solutions.

        Run the script from a cron.

        However have the script first try to grab a lock, if it can then it is the only copy running so proceed. If it cannot then exit immediately. Basically the same idea as Highlander - allow only one invocation at a time of an expensive CGI script however exiting is to be expected.

        I believe that you will find this approach significantly better for dealing with sporadic problems (lost network connections, unexpectedly high load killing a script, needing to move the script and not losing fact of the detail that it is still in init, etc). It does not solve the problem at hand though.

(tye)Re: Reasonably accurate timing
by tye (Sage) on Jul 24, 2001 at 13:40 UTC

    Can't you just do:

    $|= 1; my $next= time() + 60; while( 1 ) { print "\a"; sleep $next-time(); $next += 60; }
    There will be a very rare (I'd hope) case of a clock tick happening between the call to time and the call to sleep causing one beep to come one second late, but that won't cause the next beep to slip. When a process goes to sleep, you can't guarantee how fast it will be able to wake up anyway (that is the nature of time-share operating systems).

    If that one second turns out to be too much of a problem, then you can use the high-res equivalent:

    use Time::HiRes; $|= 1; my( $next, $us )= gettimeofday(); while( 1 ) { print "\a"; $next += 60; usleep tv_interval( [$next,$us], [gettimeofday()] ); }

            - tye (but my friends call me "Tye")
      Thanks. I used effectively the Time::HiRes version, but with sleep and time as they give good fractional values.
Re: Reasonably accurate timing
by MZSanford (Curate) on Jul 23, 2001 at 20:36 UTC
    i have, on occasion, had to use Time::HiRes for such sleep situations.
    remeber the immortal word's of Socrates who said, "I drank what ?"
Re: Reasonably accurate timing
by mattr (Curate) on Jul 24, 2001 at 13:23 UTC
    I tried a few ways, but they all seem to burp a couple of seconds once in a while even when niced. Shouldn't miss a whole minute though if you set $duration to 60.
    #!/usr/bin/perl -w use strict; use Time::HiRes qw (usleep); $|=1; my $duration = 2; # seconds between job starts my $maxsecs = 30; my $tinit = my $t0 = time; my $t1; while (1) { $t1 = time; last if $t1-$tinit > $maxsecs; if ($t1-$t0 >= $duration) { # if delta-t >= d secs &act; if ($t1-$t0 > $duration+1) { print "(",$t1-$t0-$duration,")" }; $t0 = $t1; } &waitfortick; } sub act { print "tick\n"; `boop`; # same 100% volume } sub waitfortick { print "."; #&sleepwait; # try these three separately #&selwait; &hireswait; `beep`; # from xtune, does wave -d 0.01 -f 250 -l 30 } sub sleepwait { # tick approx 1 sec sleep 1; } sub selwait { select(undef, undef, undef, 0.10); # wait approx 100 millisecs } sub hireswait { usleep 100000; # sleep 100,000 microsecs if you have usleep }
Re: Reasonably accurate timing
by thor (Priest) on Jul 24, 2001 at 16:26 UTC
    What I don't understand is why such accurate timing is required. Is you sample data only valid every 60 seconds exactly? If so, then you need to be cafeful about the thing executing every 60 seconds. If not, then 60.5 is not so bad. Time is arbitrary. Some guy a long time ago said "Well, I think a second should be this long", and it has stuck. We've standardized it since then, but it's still arbitrary. Just something to think about...
      IMHO, being half a second off doesn't matter in this situation (measuring "system data"). But if the situation were different (scientific data) a fraction of a second could be very important.

      In this situation the main thing that needs to be avoided is drift. If you want 60 second intervals, but you have 60.5 second intervals, those 0.5 seconds will add up and you will creep away from where you want to be. In an hour, it'll be off by 30 seconds, etc... What you need is some way of compensating (see all the good solutions above) so that the 0.5 second error doesn't accumulate.

(MeowChow) Re: Reasonably accurate timing
by MeowChow (Vicar) on Jul 25, 2001 at 00:54 UTC
    If your primary goal is to avoid drift, then this routine will do the trick (within the limitations of the system clock):
      
    $|= 1; my $i; my $start_time = time; my $period = 60; while (1) { print time, "\n"; my $next_sleep = ($start_time + $period * ++$i) - time; sleep $next_sleep; }
    This guarantees that future samples won't compound the timming imprecisions of prior samples.
       MeowChow                                   
                   s aamecha.s a..a\u$&owag.print