Try this.
#! perl -slw use strict; use threads qw[ yield ]; use threads::shared; use Thread::Queue; use Win32::OLE qw[ in with ]; my $Qwork = new Thread::Queue; my $Qresults = new Thread::Queue; sub worker { my $fso = Win32::OLE->new( 'Scripting.FileSystemObject' ); Win32::OLE->Option( Warn => 0 ); while( my $folder = $Qwork->dequeue ) { last if $folder eq 'DIE_NOW'; my @folders = $fso->GetFolder( $folder ); my $size = $folders[0]->size() || 'Permission denied'; my( $cFiles, $cFolders ) = (0) x 2; while( my $sub = pop @folders ) { $cFiles += $sub->Files->Count || 0; $cFolders += $sub->SubFolders->Count || 0; for my $subsub ( in $sub->SubFolders ) { $cFolders += $subsub->Files->Count || 0; push @folders, $_ for in $subsub->SubFolders; $cFolders += $subsub->SubFolders->Count || 0; } } $Qresults->enqueue( "$folder - size: $size Files: $cFiles Fold +ers: $cFolders" ); yield unless $Qwork->pending; } undef $fso; return 1; } my @threads = map{ threads->new( \&worker ) } 1 .. @ARGV; $Qwork->enqueue( @ARGV ); yield until $Qresults->pending(); for( 1 .. @threads ) { print $Qresults->dequeue; } $Qwork->enqueue( 'DIE_NOW' ) for 1 .. @threads; $_->join for @threads;
Supply the directorys/shares to be sized on the command line and it will print out lines like
P:\test>296404 M:\ S:\ C:\test C:\test - size: 22582263 Files: 7611 Folders: 352 M:\ - size: 47353164 Files: 995 Folders: 1421 S:\ - size: 99468262 Files: 1994 Folders: 3437
Not very neat, but all the info is there.
Although you cannot share an instance of Win32::OLE between threads, you can safely use a separate instance on each thread.
You might also look up the Win32::OLE->Option() class method and the Warn => n setting. This allowed me to do away with the eval wrappers you had around the OLE stuff which probably wasn;t helping your performance. Though the cost of the eval itself was probably minimal, avoiding the calls to Carp/Caller/stack trace generation is worth having.
What it does is spawns a separate thread for each share to be scanned, posts the targets on a the work queue and waits for the results to come back via the results queue and then prints them out. This allows the scans of the separate machines (and the associated IOWaits) to be overlapped.
The process consumes around .7MB of ram for each path with a 4 MB startup cost, so you should be good for a hundred scans simulataneaously. If you have more than that, it is an easy change to queue the first 100 and then queue another as each completes until your done.
I wouldn't normally advocate creating anything like this number of threads, but in this case, as each thread is essentially just sitting in a IO block waiting on other machines to respond, it makes sense to overlap as much of that waiting as possible.
Let me know how you get on please...
In reply to Re: Re: Re: Threads problem.
by BrowserUk
in thread Threads problem.
by blackadder
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |