Try this.

#! perl -slw use strict; use threads qw[ yield ]; use threads::shared; use Thread::Queue; use Win32::OLE qw[ in with ]; my $Qwork = new Thread::Queue; my $Qresults = new Thread::Queue; sub worker { my $fso = Win32::OLE->new( 'Scripting.FileSystemObject' ); Win32::OLE->Option( Warn => 0 ); while( my $folder = $Qwork->dequeue ) { last if $folder eq 'DIE_NOW'; my @folders = $fso->GetFolder( $folder ); my $size = $folders[0]->size() || 'Permission denied'; my( $cFiles, $cFolders ) = (0) x 2; while( my $sub = pop @folders ) { $cFiles += $sub->Files->Count || 0; $cFolders += $sub->SubFolders->Count || 0; for my $subsub ( in $sub->SubFolders ) { $cFolders += $subsub->Files->Count || 0; push @folders, $_ for in $subsub->SubFolders; $cFolders += $subsub->SubFolders->Count || 0; } } $Qresults->enqueue( "$folder - size: $size Files: $cFiles Fold +ers: $cFolders" ); yield unless $Qwork->pending; } undef $fso; return 1; } my @threads = map{ threads->new( \&worker ) } 1 .. @ARGV; $Qwork->enqueue( @ARGV ); yield until $Qresults->pending(); for( 1 .. @threads ) { print $Qresults->dequeue; } $Qwork->enqueue( 'DIE_NOW' ) for 1 .. @threads; $_->join for @threads;

Supply the directorys/shares to be sized on the command line and it will print out lines like

P:\test>296404 M:\ S:\ C:\test C:\test - size: 22582263 Files: 7611 Folders: 352 M:\ - size: 47353164 Files: 995 Folders: 1421 S:\ - size: 99468262 Files: 1994 Folders: 3437

Not very neat, but all the info is there.

Although you cannot share an instance of Win32::OLE between threads, you can safely use a separate instance on each thread.

You might also look up the Win32::OLE->Option() class method and the Warn => n setting. This allowed me to do away with the eval wrappers you had around the OLE stuff which probably wasn;t helping your performance. Though the cost of the eval itself was probably minimal, avoiding the calls to Carp/Caller/stack trace generation is worth having.

What it does is spawns a separate thread for each share to be scanned, posts the targets on a the work queue and waits for the results to come back via the results queue and then prints them out. This allows the scans of the separate machines (and the associated IOWaits) to be overlapped.

The process consumes around .7MB of ram for each path with a 4 MB startup cost, so you should be good for a hundred scans simulataneaously. If you have more than that, it is an easy change to queue the first 100 and then queue another as each completes until your done.

I wouldn't normally advocate creating anything like this number of threads, but in this case, as each thread is essentially just sitting in a IO block waiting on other machines to respond, it makes sense to overlap as much of that waiting as possible.

Let me know how you get on please...


Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail


In reply to Re: Re: Re: Threads problem. by BrowserUk
in thread Threads problem. by blackadder

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.