in reply to creating unknown number of threads and then join results

maybe locking the variable before I update it? I was thinking that each thread is updating a different hash key of the variable, so it should be safe to update this way? Or does it need to be locked before each join statement?

All your updates to %Found are done within the same thread, so there is no need to lock anything. Besides which %Found isn't a shared variable, so you couldn't lock it if you tried.

Any suggestions on how to do this better?

Apart from this:

foreach my $dir ( sort keys %Found ) {
   foreach my $item ( sort @{ $Found{$dir} } ) {
     push(@Final, $item);
   }
}

[download]

Could be more efficiently written as:

foreach my $dir ( sort keys %Found ) {
   push(@Final, sort @{ $Found{$dir} });
}

[download]

Not really. It is hard to see any scope for you not getting all the results produced by the external commands.

Perhaps you could print out the size of @results before returning and then sum those and compare it with the size of @Final?

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^2: creating unknown number of threads and then join results
by rudds_perl_habit (Novice) on Jul 30, 2013 at 16:36 UTC

    Thanks for the suggestion about dumping the @results array. That did help. Well, sort of. It made me more confused, actually. In the find_thread routine, I added a line before the return:

    print "find_thread dump $dir: " . Dumper(@results) . "\n";

    I then run my script 10 times dumping the results to a 10 files. What I am finding is that the dump of @results can sometimes have output that is from another directory entirely. For example:

    find_thread dump /vobs/doc: 
$VAR1 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/0';
$VAR2 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/1';
$VAR3 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/2';
$VAR4 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/3';
SIZE /vobs/doc: 4

find_thread dump /vobs/drs: 
$VAR1 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/0';
$VAR2 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/1';
$VAR3 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/2';
$VAR4 = '/vobs/license_manager/install/LMInstall/LM_Install.iap_xml@@/
+main/v4.0.0_gxp_patch/3';
SIZE /vobs/drs: 4

    [download]

    Which is totally confusing. @results is local to find_thread and shouldn't know anything about the other threads. What is even weirder is that when I switch to not use ClearCase find, and just find directories on the system, it seems to all work fine. So at this point, I am thinking that spawning multiple ClearCase find commands at once is causing an issue. I'll take it up with IBM.

[reply]
[d/l]
[select]
      So at this point, I am thinking that spawning multiple ClearCase find commands at once is causing an issue. I'll take it up with IBM.

      I concur. There is nothing in your code that could account for the symptoms you are seeing, so their source can only lie with the commands you are calling.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

        I think I stumbled onto the answer. According to this article, the chdir command is not thread-safe. See http://www.perl.com/pub/2002/06/11/threads.html. See the section about 2/3 the way down titled "Altering Perls behavior to be thread-safe, ex::threads::cwd" . They give an example of how to "fix" it, too. In my script, I use "chdir" only if running the ClearCase commands, which explains why it is failing. I'll have to re-examine how I am doing that, or implement the fix in the article above.

[reply]

In Section Seekers of Perl Wisdom