thcsoft has asked for the wisdom of the Perl Monks concerning the following question:

hi all,
i'm working on a server process, which, for various reasons, has to be multithreaded. on of these threads has quite a lot to do, but only every second: 
while (1) {
   sleep 1;
   ...
}

[download]
the thing with this thread now is, that it has to start performing at least very close to every 1 second, and not to let that value be influenced by the work it has to perform. the answer to that problem might be another thread: 
while (1) {
   sleep 1;
   my $thread = threads->new(\&work_sub);
}

[download]
i've just been testing this construct with a simple example: 
#!/usr/bin/perl -w

use threads;

while (1) {
   sleep 2;
   my $thread = threads->new( sub { print "foo"; sleep 1; print "\n" }
+);
   $thread->detach;
}

[download]
one should assume, that this program sleeps for 2 seconds, prints "foo", sleeps for another second, then prints "\n" and goes to sleep again. but what it does is sleeping for 3 seconds at first, then printing "foo\n", and falling asleep for another 2 seconds...
what startles me even more is that $thread->detach seems to have no influence at all. if commented out, the program behaves like before.

in perlthrtut it says: 
join() does three things: it waits for a thread to exit, cleans up 
after it, and returns any data the thread may have produced.
But what if you're not interested in the thread's return values,
and you don't really care when the thread finishes? All you want is 
for the thread to get cleaned up after when it's done.

In this case, you use the detach() method.  Once a thread is 
detached, it'll run until it's finished, then Perl will
clean up after it automatically.

[download]
the latter is apparently not the case with my example.
since this is a vital problem for my current project, any productive comments will be highly appreciated.
many thanx in advance. :)

language is a virus from outer space.

Replies are listed 'Best First'.
Re: a question on threads
by BrowserUk (Patriarch) on May 11, 2005 at 03:40 UTC

    Rather than starting a new thread each time you might be better using a pool of long running threads that trigger from a time taken from queue.

    The number of threads required to ensure prompt service will depend upon the time they take to do what they have to do. If the time is always less than 1 second, then you could get away with a pool of 1, but if they might take longer, you'll need more:

    Read more... (2 kB)

    Note how in the second run above, there were not enough threads running for the delay and so they gradually get further and further out of sync each timeslot. And also, how adding one more thread brings that back in the third run.

    Also, you'll get better accuracy by sleeping for a smaller time period and checking whether the timeslot has arrived than sleeping for an absolute time. The smaller the sleep, the greater the accuracy, but it will still never be always exact (within some small descrepancy) because you have to wait for the scheduler to give the appropriate thread a timeslice.

    Also, reducing the sleep below a certain minimum will greatly increase the cpu time spent polling. A tenth of a second works well on my system, giving an accuracy of < 1/10th whilst burning negligable amounts of cpu.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
[reply]
[d/l]
      thank you very much for your precious advice, and the example code. to give you an update on my code, as i've done it so far: 
      sub event_handler {
   my $worker_thread = sub { 
          my $row = shift;
          my $event = SOD::EventCooker->new($row);
          if (exists $p_ids{$row->{'ev_p_id'}}) {
             lock $sessions{$p_ids{$row->{'ev_p_id'}}};
             my $session = thaw sessions{$p_ids{$row->{'ev_p_id'}}};
             $event->evaluate($session->{PLAYER}->{GAMESTATS});
             $sessions{$p_ids{$row->{'ev_p_id'}}} = freeze $session;
          }
          else {
             $event->evaluate;
          }
      };

   my $dist_thread = sub { 
          my @threads;
          my $thread_time = time;
          my $dbh = DBI->connect("DBI:mysql:database=$appdata{DBBASE}"
+, $appdata{DBUSER}, $appdata{DBPASS},
          { RaiseError => 1, AutoCommit => 1 }) || die $dbh->errstr;
          my $res = $dbh->selectall_hashref("SELECT * FROM event WHERE
+ ev_time<=$thread_time", 'ev_time');
          $dbh->do("DELETE FROM event WHERE ev_time<=$thread_time"); 

          map {
              push @threads, threads->new($worker_thread->($res->{$_})
+);
          } sort keys(%$res);
          map {
             $_->detach;
          } @threads;
          $dbh->disconnect;
          undef $dbh;
      };
   
   while (1) {
      sleep 1;
      last if $tflag;
      threads->new($dist_thread)->detach;
   }   
}

      [download]
      i assume that a combination of our two approaches might be sufficient even for time of high traffic & loads.

      language is a virus from outer space.
[reply]
[d/l]
        assume that a combination of our two approaches might be sufficient even for time of high traffic & loads.

        It really depends upon how you combine them :)

        What I can say is that your current code that starts two threads every second with one of them making a new connection to a database each time and the other freezing and thawing a compound structure to and from a shared hash is not going to run quickly.

        Perl's threads are quite different from the kind threads found in say Java or Ruby where this kind of "spawn a thread, do a little and throw it away" coding style is common. and practical. Even there, creating new connections to MySql at the rate of one per second and then abandoning them is very likely to consume (leak) resources both in the DBI layers and at the MySQL server end.

        I seem to recall that (at v3.xomething), MySQL would not reuse a dropped connection for something like 15 minutes or so? This may have been fixed or be configurable or I may have remembered it wrong, but in any event it is a far from ideal strategy if you are hoping to handle high data rates.

        Using a single, long running thread that makes the connection to the DB when it starts and then reuses the handle to issue queries is a much better idea. Without seeing where, how and when your event_handler routine is called; and what SOD::EventCooker is; and what is populating your 'event' table, it is quite hard to advise further.

        A high level pseudo-code description of your app might allow us to advise further.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
[reply]
      ok, i modified my code according to your suggestions. i put the snippet on http://no-subject.org/event_handler.snippet (because i'm too lazy re-editing it here. ;)
      i haven't tested it yet, so there might be some smaller bugs, but in principle, it represents the compromise we were talking about.

      many thanks for your help! :)

      language is a virus from outer space.
[reply]
Re: a question on threads
by davidrw (Prior) on May 11, 2005 at 02:30 UTC
    Is it just a matter of buffering where the "foo" really is being printed 1 second before the "\n" and you're just not seeing it flushed? i.e., try putting $|=1 in your script and seeing if you get teh expected 2s-foo-1s-\n behavior.

    side comment -- I'm not too familiar with timings/accuracies of sleep, but Time::Hires might be more accurate if exactly 1 second (or 2 or 3) is crucial to your application.
[reply]
[d/l]
Re: a question on threads
by Errto (Vicar) on May 11, 2005 at 02:41 UTC
    I agree with davidrw that the specific problem you're seeing is probably just output buffering - that is, the print "foo" really does happen when you think it should it just doesn't look like it - but here's another issue to consider: what happens if the job to be done takes more than one second? Is it ok to have more than one thread performing this action at the same time?
[reply]
[d/l]
      yes, it will be ok, because each of them will have an individual set of data to act upon. nevertheless: thank you for the $| hint :)
      language is a virus from outer space.
[reply]
Re: a question on threads
by djp (Hermit) on May 11, 2005 at 05:33 UTC
    alarm rather than sleep might provide your solution.
[reply]

      Do signals work with threads?

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
[reply]
[d/l]
        Possibly not, hence 'might'.
[reply]
        Not unless you feel like having your signal sent to every thread of your process.
[reply]
Re: a question on threads
by salva (Canon) on May 11, 2005 at 12:58 UTC 
    use Time::Hires qw(time usleep);

while(1) {
  my $now = time;
  my $next = int($now+1);
  usleep $next-$now;

  foo; bar; # your work here!
}

    [download]
[reply]
[d/l]
      If you did the work first and did the sleep after, you wouldn't have slippage induced by the the time it takes to do the work.
      Caution: Contents may have been coded under pressure.
[reply]
        that doesn't matter. i'm afraid. the actions have to be taken nearly every 1 second. so, the tiome, when the thread falls asleep, won't be of any effect - as long as it sleeps for a while as close to a second as possible.

        language is a virus from outer space.
[reply]

Back to Seekers of Perl Wisdom