Hi, I think you are confusing defunct processes with threads. When using threads, the spawned threads can either be joined or detached. If you detach it, it dosn't need to be joined, it will cease when the code block ends. When the parent thread exits, it will take all child threads with it, and issue a warning that some child threads were running at exit.

As usual, you should should make a complete self-contained running example. Here are a few to show the difference between detach and join.:

#!/usr/bin/perl
use strict;
use warnings;
use threads;

foreach (0..100){
  # create a thread to call the "run_system" subroutine
  # to run "ls -la" on the OS
  my $thread = threads->create("run_system", "ls -la");
   $thread->detach;
 }

sub run_system{
    my $cmd = shift;
    system($cmd);
}

__END__
#1liner
perl -Mthreads=async -le"async{ system 'dir'; } for 1 .. 100"
[download]

#!/usr/bin/perl
use warnings;
use strict;
use threads;

# join() does three things: it waits for a thread to exit,
# cleans up after it, and returns any data the thread may
# have produced. 

my $thr1 = threads->new(\&sub1);
my $ReturnData1 = $thr1->join;
print "Thread1 returned @$ReturnData1\n";

my $thr2 = threads->new(\&sub2);
my $ReturnData2 = $thr2->join;
print "Thread2 returned @$ReturnData2\n";

my $thr3 = threads->new(\&sub3);
my $ReturnData3 = $thr3->join;
print "Thread3 returned @$ReturnData3\n";


sub sub1 {
print "In thread1.....\n";
sleep 5;
my @values = ('1a','1b', '1c');
return \@values; 
}

sub sub2 {
print "In thread2.....\n";
sleep 5;
my @values = ('2a','2b', '2c');
return \@values; 
}

sub sub3 {
print "In thread3.....\n";
sleep 5;
my @values = ('3a','3b', '3c');
return \@values; 
}
[download]

In preparation of trying to suggest a replacement for your switch use i would like to start with this comment

$t[++$#t]=threads->new(\&adminpass); seems to be a real verbose way of saying push @t,threads->new(\&adminpass);

It looks like your use of switch is to process a comma separated list of commands. a simple replacement for

   switch($action) {
      case /adminpass(,|$)/ {
         print "Do adminpass ($hostname)\n" if($verbose || $debug);
         $t[++$#t]=threads->new(\&adminpass);
         next;
      } # end case /adminpass/
      case /backup(,|$)/ {
         print "Do backup ($hostname)\n" if($verbose || $debug);
         $t[++$#t]=threads->new(\&backup);
         next;
      } # end case /backup/
      case /buildhosts(,|$)/ {
         print "Do buildhosts ($hostname)\n" if($verbose || $debug);
         $t[++$#t]=threads->new(\&buildhosts);
         next;
      } # end case /buildhosts/
      case /buildhostsdell(,|$)/ {
         print "Do buildhostsdell ($hostname)\n" if($verbose || $debug
+);
         $t[++$#t]=threads->new(\&buildhostsdell);
         next;
      } # end case /buildhostsdell/
...
}
[download]

      if ($action=~/adminpass(,|$)/) {
         print "Do adminpass ($hostname)\n" if($verbose || $debug);
         push @t,threads->new(\&adminpass);
      } # end case /adminpass/
      if ($action=~/backup(,|$)/) {
         print "Do backup ($hostname)\n" if($verbose || $debug);
         push @t,threads->new(\&backup);
      } # end case /backup/
      if ($action=~/buildhosts(,|$)/) {
         print "Do buildhosts ($hostname)\n" if($verbose || $debug);
         push @t,threads->new(\&buildhosts);
      } # end case /buildhosts/
      if ($action=~/buildhostsdell(,|$)/) {
         print "Do buildhostsdell ($hostname)\n" if($verbose || $debug
+);
         push @t,threads->new(\&buildhostsdell);
      } # end case /buildhostsdell/
...
[download]

 
# create a list of the command names and the subroutines they call 
my %cmds=(adminpass      =>\&adminpass,
          backup         =>\&backup,
          buildhosts     =>\&buildhosts,
          buildhostsdell =>\&buildhostsdell
          ); 
# break list of comma sep commands into words 
#  and if we have a subroutine for that word start a thread with it  
for my $cmd (split(',',$action)) { 
      if (exists $cmds{$cmd}) { 
         print "Do $cmd ($hostname)\n" if($verbose || $debug);
         push @t,threads->new($cmds{$cmd});
      } 
      else { 
        print "Command $cmd unknown\n"; 
      } 
}
[download]

While i am far from truly knowing much about these things i agree with zentara's statement I think you are confusing defunct processes with threads. I believe defunct process are the result of exec, fork or certain types of system calls. It is those that you have to wait on. I think to solve this problem we will need to see the areas of your code that use those rather than where you start threads.

Someone else more knowledgeable than me will probably also pipe up about the fact that the use of switch is discouraged. While it MAY not be the cause of your current fire you may want to look into ways to replace its use with more "modern" perl code

Another sketch for further inspiration:

#!/usr/bin/env perl

#     $Id: async.pl,v 1.8 2017/07/03 19:46:43 karl Exp karl $

use strict;
use warnings;
use threads;
use MCE::Hobo;
use feature qw(say);

my %actions = (
    nose => sub {
        my $hobo = mce_async { shift } "kumquat";
        ( $hobo->pid, $hobo->join );
    },
    foo => sub {
        my $hobo = mce_async { shift } "bar";
        ( $hobo->pid, $hobo->join );
    },
);

say join " ", $actions{q(nose)}->();
say join " ", $actions{q(foo)}->();

__END__
[download]

Please note: if you say # use threads it forks.

This is not quite correct:

"...Only the shared-server is a thread when loading threads. Hobo workers are spawned via fork even when loading threads." marioroy

See also MCE::Hobo

Best regards, Karl