blackadder has asked for the wisdom of the Perl Monks concerning the following question:

Hi Guys

I have this simple code below, all it does is,
1- Gets the shared areas of a remote machine.
2- Obtains the top level folder of that share and the drive letter where the share resides.
3- Finally it obtains the size of that top level folder.

However, the script never completes its execution, it through up a message out of memory

Does any of you holly ones knows how can I get this script to work?.....

Many Thanks and sorry for the style of my coding.
use strict; use Win32::Lanman; use File::Spec qw (splitdir); my $input = shift @ARGV; push (my @SrvLst, $input); for my $Srv (@SrvLst) { &Get_Shr($Srv); print "___________________________________________\n"; } sub Get_Shr { my $Machine = shift @_; my $UncSrvName = "\\\\" . $Machine; print "\nAttempting to access '$UncSrvName'\n"; if (Win32::Lanman::NetShareEnum( $UncSrvName , \ my @EnumLst)) { my %SeenPath=(); for my $Share (@EnumLst ) { next if ( ($Share->{path}) =~ /^c/i); next if ( ($Share->{netname})=~ /ipc\$|rpc\$|netlogon\$|ad +min\$|^CDROM|^\w\$/i); my @SplitDir = File::Spec->splitdir(uc ($Share->{path})); my $DrvLttr = @SplitDir[0]; my $TopDir = @SplitDir[1]; my $DrvTop = $DrvLttr . "\\" . $TopDir; unless ( ($SeenPath{($DrvTop)})) { $SeenPath{$DrvTop}=1; print "\nDrvTop=> $DrvTop "; $DrvLttr =~ s/:/\$/; my $TargetPath = $UncSrvName . "\\". $DrvLttr ."\\" . + $TopDir; print "\tSizing "; if (opendir ( DIR, "$TargetPath")) { print "$TargetPath"; my ($Size)=( (qx{dir /s "$TargetPath"})[-2]=~ /([\ +d,]+) bytes/ ); print " : $Size\n"; undef $Size; } else { warn " \n(Access Error : $^E)\n"; } } } } else { print Win32::FormatMessage(Win32::Lanman::GetLastError()) . " +$Machine\n"; } }

Replies are listed 'Best First'.
Re: Out of memory.
by BrowserUk (Patriarch) on Jul 22, 2003 at 16:33 UTC

    The problem is that you are shelling out and doing a dir /s of an entire drive in order to extract the size from the second from last line.

    Even on my half empty 2GB boot partition, this results in 1.5 MB of data and 36,000 lines!

    On a share that points to a well used 40GB drive with compression enabled, this could easily lead to a list of files of several 10s of MB and 200,000 lines or more. Pulling all this data across the network and into an anonymous list just to find out the amount of space used on a remote share is ... well more than just slightly profligate.

    There are a number of vastly more efficient ways (in terms of memory, time and network load) of getting this information. The question has comes up at least three times in the last week or so, including this one. A Super search for "disk usage" or "share size " should turn up several more.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller

      Thanks for that, but its the folder sizes is what I am after, not the drive sizes, I do get the drive sizes via a combination of AdminMisc and Netadmin, but the OLE->GetDrive method you have suggested is definitely much faster.

      I have searched for sometime for a remedy for this problem to no avail…The struggle shall continue.

        I hate suggesting this, but it will probably make the out-of-memory problem go away with the least effort:)

        Change

        my ($Size)=( (qx{dir /s "$TargetPath"})[-2]=~ /([\d,]+) bytes/ );

        to

        my ($Size)=( (qx{dir /s "$TargetPath" | find "bytes" })[-2]=~ /([\d,]+ +) bytes/ );

        That will filter out the vast majority of the lines produced by the dir /s before they get into perl.

        If you have WSH available, then you can use

        use Win32::OLE; my $size = Win32::OLE->CreateObject('Scripting.FileSystemObject') ->GetFolder( $TargetPath ) ->size();

        Which probably isn't hugely more efficient than dir /s in terms of doing the calculation, but it will prevent the memory problem.

        You could also accumulate the directory sizes directly in perl

        my (@dirs, @files) = ($TargetPath); scalar map{ push @{ (-d) ? \@dirs : \@files }, $_ } glob pop(@dirs) . '/*' while @dirs; my $size += -s for @files;

        Which from my quick tests is much faster than shelling out, and seems to be quicker than using an FSO but it's difficult to measure as file system catching gets involved.

        You could also use any of the many File::Find modules to achieve the same effect.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller

Re: Out of memory.
by dragonchild (Archbishop) on Jul 22, 2003 at 15:28 UTC
    Given that you have a bunch of print statements, it would be interesting (and kinda helpful if you want us to help) to know where the print statements stop. (A copy of the output would help ...). If you don't see any, put
    $|++;
    at the top of your script. That will force output to be sent to the screen immediately, vs. being buffered.

    I would also add a print statement immediately within the if (Win32::Lanman::NetshareEnum( ... )) { ... } block, just to make sure your memory issue isn't in that function call.

    ------
    We are the carpenters and bricklayers of the Information Age.

    Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

    Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

      "$|++"; I will make a habit of adding this from now on, and I will try the print statement within the if (lanman...)
      ...cheers.
Re: Out of memory.
by naChoZ (Curate) on Jul 22, 2003 at 15:08 UTC
    Don't forget

    use warnings; use diagnostics;

    You may also use -T to enable taint checking and see if that tells you anything interesting.

    ~~
    naChoZ