blackadder has asked for the wisdom of the Perl Monks concerning the following question:

Greatings Respectable Monks....

This is my first attemp in multi-threading so please bear with me.

I have this code 
$|++;
AUTOLOAD;
use strict;
use warnings 'all';
use diagnostics;
use Win32::OLE qw[in];
use threads;
use threads::shared;
my $fso = Win32::OLE->new( 'Scripting.FileSystemObject' );
my @childs;
my @up: shared; 
for (@ARGV){
    print "\n$_\n";
    push @childs, threads->create("Process_path","$_");
    #Process_path($_);
}
foreach my $child (@childs){
    $child->join();
}
print @up;
sub Process_path{
    my ($path) = @_;
    my @folders = $fso->GetFolder( $path);
    my $FSize = $fso->GetFolder($path);
    my $Size = $FSize->size();
    print "Size: $Size\n";
    my $fCount =0;
    my $sCount =0;
    while( @folders ){
        my $folder = pop @folders;
        $fCount += $folder->Files->Count; 
        $sCount += $folder->SubFolders->Count; 
        for my $subFolder ( in $folder->SubFolders ) {
            $fCount += $subFolder->Files->Count;
            push @folders, $_ for in $subFolder->SubFolders ;
            $sCount += $subFolder->SubFolders->Count;
        }}
    print "Files : $fCount, folders : $sCount\n";
    push (@up, $path.": ".$Size.": ".$fCount.": ".$sCount.".");
}

[download]
which is the result of marrying This script, that obtains the number of files and folders for a given path, to This one, that uses a multi-threading technique to ping few servers, so that instead of pinging servers the script will obtain the size, number of files and folders of a list of folders/paths/share…etc.

To me, all I have done is replaced the pinging off server with obtaining the file/folders statistics. But I am getting this error; 
  
Free to wrong pool 1a9e940 not 223f80 at C:\Scripts\threads4_wsh.pl li
+ne 29.
Free to wrong pool 1ab0998 not 223f80 at C:\Scripts\threads4_wsh.pl li
+ne 15.

[download]
Since this my first attemp with multithreading, I am not sure how to fix it.

Your help and Perls of wisdom, as ever, are higly appreciated.

Thanks.

Replies are listed 'Best First'.
Re: Starting multi-threading
by PodMaster (Abbot) on Aug 24, 2003 at 14:25 UTC
    I'm guessing (and this is a pretty good guess) that you can't share a Win32::OLE object. Try creating it in sub Process_path. Also, read the Win32::OLE documentation and look for the word "thread".

    BTW, you got a stray AUTOLOAD; up top.

    update: I just noticed that you're not actually sharing $fso, but are hoping that somehow automagically $fso and all it's underlying structures will get copied -- it doesn't work like that (especially for XS stuff).

    update: http://listserv.activestate.com/pipermail/perl-win32-users/2002-September/023498.html

    MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
    I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
    ** The third rule of perl club is a statement of fact: pod is sexy.

[reply]
Re: Starting multi-threading
by BrowserUk (Patriarch) on Aug 24, 2003 at 15:12 UTC

    PodMaster is right. You cannot share objects across threads.

    The answer may be to create a new instance of the scripting object in each thread. I haven't tried this, so you'll have to suck it and see, but I don't see any reason why it wouldn't work.

    As for the free to wrong pool. Do you get these messages when the program closes down?

    One note of caution. Multi-threading directory scans, especially if the all (or many of) the paths are on the same physical drive, will have rapidly diminishing returns in terms of performance. Each time a context switch occurs--ie. one thread gets switch for another by the scheduler--then the drive head will have to move to another part of the disk to scan the path for that thread, and then move somewhere else as the next thread takes its turn.

    Moving heads between tracks is hugely more expensive than just waiting for the disk to roll passed the head, so the penalty can outweight the benefits. Additionally, reading another part of the disc can cause data already read from the disc into cache but not yet supplied to the program to be invalidated and discarded so that when the program does get around to asking for that information it has to be re-read.

    Don't let that put you off trying, but don't be surprised if you find that whilst two or three threads improves performance that 4 starts to diminish it, and 5 really slows things up.

    NOTE: Those numbers are just examples. The breakpoint could 6 or 8 or 2. Even just 1, but normally not.

    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
    If I understand your problem, I can solve it! Of course, the same can be said for you.
[reply]
      ok, this is what I have changed. 
      AUTOLOAD;
require 5.008;
use strict;
use warnings 'all';
use diagnostics;
use Win32::OLE qw[in];
use threads;
use threads::shared;
use strict;
my @up: shared; 
my @childs;
my $child;
my @down: shared;
my @list = qw ( c:/winnt d:/perl );
for (@list){
    push @childs, threads->create("process_path","$_");}
for $child (@childs){
    $child->join();}
for my $info (@up){
 print $info;}#I'll do some work here once I get results
sub process_path{
    my $path = shift @_;
    my $Details={};
    my $fso = Win32::OLE->new( 'Scripting.FileSystemObject' );
    my $FSize = $fso->GetFolder($path);
    my $Size = $FSize->size();
    $Details->{path}=$path;
    $Details->{size} =$Size;
    push @up, $Details;
}

      [download]
      and this what iget back in return. 
      C:\Scripts>threads1.pl
Win32::OLE(0.1403) error 0x800a004c
Win32::OLE(0.1403) error 0x800a004c
    in METHOD/PROPERTYGET "GetFolder" at C:\Scripts\threads1.pl line 3
+3
thread failed to start: Can't call method "size" on an undefined value
+ at
        C:\Scripts\threads1.pl line 34 (#1)
    (F) The entry point function of threads->create() failed for some 
+reason.

    in METHOD/PROPERTYGET "GetFolder" at C:\Scripts\threads1.pl line 3
+3
thread failed to start: Can't call method "size" on an undefined value
+ at
        C:\Scripts\threads1.pl line 34 (#1)
    (F) The entry point function of threads->create() failed for some 
+reason.

Free to wrong pool 1a99e90 not 223f58 during global destruction.

      [download]
      I don't know what else to do to it! So, is this the end?

      Cheers.
[reply]
[d/l]
[select]

        Sorry! Bad guess. I did warn you I hadn't tried it.

        Unfortunately, whilst the scripting object itself and OLE are reentrent, the means of getting to them from perl is not. Any attempt I make to use Win32::OLE from two threads at the same time, even if it is to two different OLE objects, the second one fails.

        I took a quick scan at Win32::OLE but it doesn't look like it would be an easy thing to fix. It was written before threads were around, so reentrancy was never a consideration.

        I had thought that by creating new instances in each thread, which under win32 is pretty much the same as creating different instances in seperate (pseudo-) processes, that it would isolate everything, but it would appear not.

        As I said in the last post, I'm not sure how much you would have gained by threading for this anyway, but unless Win32::OLE gets an update some time soon, I don't see this being easily fixed.

        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
        If I understand your problem, I can solve it! Of course, the same can be said for you.
[reply]

Back to Seekers of Perl Wisdom