in reply to Re^12: does threads (still) memleak?
in thread does threads (still) memleak?

Here's the big picture: ... periodically store some time-series numbers ... and with its own interval

As overviews go, that's a little like looking at the moon through the wrong end of a telescope. You kinds know what you're looking at, but the details decidedly fuzzy :)

A few things that would clarify:

I'd need a far clearer picture of the task before I'd be prepared to offer a design for it's solution.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^14: does threads (still) memleak?
by faxm0dem (Novice) on Nov 28, 2008 at 14:11 UTC
    • How is the shared hash keyed?

      Each thread feeds a seperate key e.g. $h->{thX}

    • What does "time-series" numbers" mean? 
      $h->{th1}->[[ts1,d11,...,d1N],...,[tsM,dM1,...,dMN]]
$h->{th2}->[[ts1',d11',...,d1K'],...,[tsL',dL1',...,dLK']]

      [download]

      each thread has its own independant [tsI,[dIJ]] values

      tsI are timestamps

      dIJ are data (i.e. temperature, cpu cycles, memory usage...)

    • When the sending thread wakes up to take it copy, does it just take whatever is available, or should it only take a "complete set"?

      It takes whatever is ready, and each [tsI,dI1,...,dIN] should be complete.

    As for the timeslices, each thread handles its own, and yes they are pretty long - at least 15 seconds.
[reply]
[d/l]
[select]

      Okay. On the basis of the (still scant) specification provided, this is how I would prototype that application. You'll see that I've dodged the issue of the shared hash and locking all together by using a queue: 

      # perl -slw
use strict;
use Time::HiRes qw[ time sleep ];
use threads;
use threads::shared;
use Thread::Queue;

$|++;

my $SOsem : shared;
sub tprintf { 
    lock $SOsem; 
    printf "[%3d]" . $_[ 0 ], threads->tid, @_[ 1 .. $#_ ]; 
}
sub tprint { 
    lock $SOsem; 
    printf "[%3d] %s" . $/, threads->tid, join ', ', @_; 
}


sub gatherer {
    my $tid = threads->tid;
    my( $Q, $stop, $interval ) = @_;
    while( !$$stop ) {
        ## Get dataset
        my @data :shared = ( $tid, time(), map int( rand 100 ), 1 .. 5
+ );
        $Q->enqueue( \@data );
        sleep $interval;
    }
}

sub sender{
    my $tid = threads->tid;
    my( $Q, $stop, $interval ) = @_;
    while( sleep $interval ) {
        my $available = $Q->pending;
        last if $$stop and !$available;
        next unless $available;
        my @datasets;
        push @datasets, [ @{ $Q->dequeue } ] for 1 .. $available;

        ## 'send' the data
        tprint "\n", time();
        tprint "@$_" for @datasets;
    }
}

our $GATHERERS ||= 5;
our $INTERVAL  ||= 15;
our $GINTERVAL ||= $INTERVAL;

my $Q = new Thread::Queue;
my $stop :shared = 0;
$SIG{ INT } = sub { $stop = 1 };

threads->create( \&gatherer, $Q, \$stop, $GINTERVAL )->detach
    for 1 .. $GATHERERS;

my $sender = threads->create( \&sender, $Q, \$stop, $INTERVAL )->join;

      [download]
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
        Hi, Looks great - thanks a lot (sorry for the late feedback). I do have a small problem though: when using this as-is, SIGINT gets ignored by the script. I can get around this using an endless loop in main before the joining: 
        while (!$stop) {
  sleep 1;
}

$sender->join;

        [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom