losgrandes has asked for the wisdom of the Perl Monks concerning the following question:

my $h; my $handle = tie $h, 'IPC::Shareable', { destroy => 'Yes' }; # start remote command for each host my $pm = new Parallel::ForkManager(scalar @hosts_to_run); $pm->run_on_finish( sub { my ($pid,$exit_value,$ident) = @_; print "finished run on $ident\n" if $verbose; } ); foreach my $host (@hosts_to_run) { my $pid = $pm->start($host) and next; my $result = #some calculations $handle->shlock(); $h->{$host}{output} = $result; $handle->shunlock(); $pm->finish(0); } $pm->wait_all_children; IPC::Shareable->clean_up_all;
Above code works perfectly in terms of populating $h with data, but leaves semaphores in system after execution. Any ideas where the problem is?

Replies are listed 'Best First'.
Re: Semaphores leaks with IPC::Shareable
by zentara (Cardinal) on Oct 22, 2014 at 14:32 UTC
    I don't know if you can do this from IPC::Shareable, but you probably need to do something like this:
    # Deallocate the shared memory segment when exiting program. shmctl ($segment_id, IPC_RMID, 0); $sem->remove;
    update
    From the IPC::Shareable docs, it says for the ->destroy method: destroy
    If set to a true value, the shared memory segment underlying the data
    binding will be removed when the process calling tie() exits
    (gracefully)3. Use this option with care. In particular you should
    not use this option in a program that will fork after binding the
    data. On the other hand, shared memory is a finite resource and should
    be released if it is not needed. The default is false

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
      IPC::Shareable takes care of creating semaphores and destroying them. At least should. Well, looks like I missed that part from docs, somehow. Thanks for pointing out. I'll check it without destroy and let you know.
        The docs for destroy says that:
        Use this option with care. In particular you should not use this option in a program that will fork after binding the data.

        You fork.


        I'm not really a human, but I play one on earth.
        Old Perl Programmer Haiku ................... flash japh
Re: Semaphores leaks with IPC::Shareable
by zentara (Cardinal) on Oct 22, 2014 at 15:43 UTC
    Here is a full example which I play with. It works. Notice the line
    $parent_share->remove;
    I run the script, then afterwards run "ipcs -a" to list ALL, and everything is cleaned up. I did run into one weirdness however, and it may be a bug. If you remove the last 2 cleanup lines
    # $parent_share->remove; # $parent_share->clean_up_all;
    the script will leave behind some shmem segments that cannot be removed, even by root. Only killing the X server seems to remove it. So it seems like a zombie situation of sorts. If the running script dosn't do the cleanup, junk gets left behind. There are alot of post on google on removing shared mem segments and semaphores, and you may need to resort to using some kind of clean-up script after you run your program. See man ipcs, and man ipcrm
    #!/usr/bin/perl use strict; use Parallel::ForkManager; use IPC::Shareable; my $glue = $$; my %options = ( create => 1, exclusive => 0, mode => 0644, destroy => 1, ); my %final_parent_hash; my $parent_share = tie %final_parent_hash, 'IPC::Shareable', $glue, { +%options } or die "parent : tie failed\n"; my $fork_manager = new Parallel::ForkManager(5); foreach my $child ( 1 .. 10 ) { my $pid = $fork_manager->start($child) and next; my %options = ( create => 0, exclusive => 0, mode => 0644, destroy => 0, ); my %child_hash; my $child_share = tie %child_hash, 'IPC::Shareable', $glue, { %opt +ions } or die "client: tie failed\n"; for my $id (1 .. 20) { my $key = $child . '-' . $id; $child_share->shlock; $final_parent_hash{$key} = qq{|Kid $child pushed $id}; $child_share->shunlock; } $fork_manager->finish($child); } print "Waiting for Children...\n"; $fork_manager->wait_all_children; my $sleep_ctr = 10; while ($sleep_ctr) { if ((keys %final_parent_hash) == 200) { last; } sleep 1; $sleep_ctr--; } foreach my $child ( 1 .. 10 ) { for my $id (1 .. 20) { my $key = $child . '-' . $id; if (! exists $final_parent_hash{$key} ) { print "Missing data for Kid $child , data $id\n"; } else { print "$key = $final_parent_hash{$key}\n"; } } } $parent_share->remove; $parent_share->clean_up_all;

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
      Ok, I've resolved the riddle. I've removed Destroy option and it was the same. Tried many solutions, including yours with 2nd tie for children, but nothing worked. Finally, I've changed tied variable from $h to %h, it worked and all semaphores were removed. Looks like it had problem with nested data/references. Thanks for help!