Hello Monks, I need some help with this problem. I am working on a perl code to run File::Find. Code is to try and spawn multiple threads that does a find on different mountpoints/dirs (finds files owned by specific uid). I want to spawn multiple threads so that I could reduce the time taken to search the whole system.
However the problem is when i spawn only one thread (ie, $thr1), it is able to find all files from that dir. However, when i enabled both threads, thread 1 is skipped in middle. 

example

When only thr1 was enabled thr2 was commented out:<br>
tstsrv:/home/aaron # ./collect_files_threaded.pl -u aaron | wc -l
3948

When only thr2 was enabled thr1 was commented out:
tstsrv:/home/aaron  # ./collect_files_threaded.pl -u aaron | wc -l
1436

When both were enabled (below code):
tstsrv:/home/aaron  # ./collect_files_threaded.pl -u aaron | wc -l
1753

Each thread updates a global variable.

I got the code generated by find2perl command to create the function f
+indFiles.

[download]
use File::Find;
use threads;
use threads::shared;

my $user :shared = $cargs{u};
my $uidNum :shared = getpwnam($user);
my $user_file :shared = "/var/tmp/test_$user.file.list";
my @array_to_write :shared;

my $thr1 = threads->create(\&find, \&findFiles, "/var/tmp");
my $thr2 = threads->create(\&find, \&findFiles, "/tmp");
$thr1->join();
$thr2->join();


print "Saving the details to $user_file\n";
saveFile($user_file,@array_to_write);

exit;

sub findFiles {
    my ($dev,$ino,$mode,$nlink,$uid,$gid,$file,$fstype);
    $file=$File::Find::name;
    
        lock(@array_to_write);
        
        
        (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($file)) &&
        !($File::Find::prune |= ($dev != $File::Find::topdev)) &&
        ($uid == "$uidNum") && 
        push (@array_to_write,"$file:$uidNum:$gid:$mode:$fstype");
    
}

sub saveFile{
    my($file,@data)=@_;
    open(F,">$file") or die("Error:Failed to open $file for write:$!\n
+");
    print F (join("\n",@data) . "\n");
    close(F);
}

[download]

======================================================================

Update: Moved the parallel execution of find into multiple processes instead of threads. Each process takes care of a mount point available in the system. 

foreach my $mnt (@mounts) {
    $script="file_collections.pl -u $user -m $mnt";
    my $pid = fork;
    if (not defined $pid) {
        die 'Could not fork child';
        next;
    }
    if ($pid) {
        $forks++;
        Log ("Parent process PID ($$) starting Child pid: $pid for $mn
+t. \nNum of forked child processes: $forks\n");
    }
    else {
        Log ("child PID ($$) --> executing \'$script\' \n"); 
        exec ("$scr_loc/$script");
        exit;
    }
}

for (my $i = 0; $i < $forks; $i++){
    my $pid = wait();
    Log ("$0 : $pid Completed\n");
}



file_collections.pl

Log ("Start Time \t\t: $datestring \n\n");
Log ("Finding the files for $user with uid $uidNum on $mount\n");

$path=$mount; 
$path=~s/\//_/g;

$user_file = "$log_dir/$user$path.list";
$user_log_file = "$log_dir/$user$path.log";

find(\&findFiles, "$mount");

$datestring = getDateTime();

Log ("Total number of files parsed on $mount = $count\n");
Log ("Saving the details to $user_file\n\n");
Log ("End Time \t\t: $datestring\n");

saveFile($user_file,@array_to_write);
saveFile($user_log_file,@log);


sub findFiles {
    my ($dev,$ino,$mode,$nlink,$uid,$gid,$file,$fstype);
    $file=$File::Find::name;
    $fstype=findFstype("$file");
    $count++;        

    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($file)) &&
    !($File::Find::prune |= ($dev != $File::Find::topdev)) &&
    ($uid == "$uidNum") && 
    push (@array_to_write,"$file:$uidNum:$gid:$mode:$fstype");

    
}

[download]

In reply to Perl threads - parallel run not working by aaron.schweitzer

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.