ChrisR has asked for the wisdom of the Perl Monks concerning the following question:

While using Win32::Perms to get file ownership information, I have run into a snag that I can't track down. Please, consider the following code: 
#!c:\perl\bin\perl.exe -w

use strict;
use warnings;
use diagnostics;

use File::List;
use Win32::Perms;

my $search2 = new File::List("c:\\");
my @tree2 = @{$search2->find(".*")};
my %ownerhash = GetFilesByUser(@tree2);

for my $key(sort keys %ownerhash)
{
    print "$key $ownerhash{$key}{numbytes} bytes in $ownerhash{$key}{n
+umfiles} files\n";
}
exit;


sub GetFilesByUser
{
    my @dirtree = @_;
    my $file_obj;
    my $fileowner;
    my %temphash;

    for my $x (0..$#dirtree)
    {
        $dirtree[$x] =~ s/\//\\/g;
        $dirtree[$x] =~ s/\\\\/\\/g;
        $file_obj = new Win32::Perms($dirtree[$x]) || print "ERROR1: $
+!\nFILE: $dirtree[$x]\n";
        if(not $file_obj)
        {
            $temphash{FILESTHATGOTERRORS}{numfiles} ++;
            $temphash{FILESTHATGOTERRORS}{numbytes} += -s $dirtree[$x]
+;
        }
        elsif($file_obj ne "1")
        {
            $fileowner = $file_obj->Owner() || print "ERROR2: $!\nFILE
+: $dirtree[$x] \n";
            $file_obj->Close();
            $temphash{$fileowner}{numfiles} ++;
            if(! -z $dirtree[$x]) #check to see if file is zero bytes
            {
                $temphash{$fileowner}{numbytes} +=  -s $dirtree[$x] ||
+ print "ERROR3: $!\nFILE: $dirtree[$x]\n";
            }
        }
    }
    return %temphash;
}

[download]
The code works. I know that it's not pretty and my error checking could use some work but that's not the problem. The script process many files per second up to a point and then it begins to take about 5-6 seconds per file. The problem starts at file number 31657 and is OK again after file number 34093. There are over 35000 files to process. What I have found is that it has nothing to do with the number of files processed. I have run this script many times on a few different machines and have found that the problem only occurs in one directory. I have also found that the extra time is consumed when I'm creating the Win32::Perms object ( $file_obj = new Win32::Perms($dirtree[$x])). The directory in question is a shared directory. The administrator, which is running the script, has full controll over the directory. Considering that there are more than 2400 files in this directory, the time it takes to process this directory is well over 3 hours while it takes less than 8 minutes to process the rest of the 350000 files. System resources are not an issue as processor and memory usage are very low while the script processes the directory in question. I just can't figure out why Win32::Perms is so slow on that one directory. Any ideas??

Replies are listed 'Best First'.
Re: Another Strange Win32::Perms Issue
by BrowserUk (Patriarch) on Oct 28, 2003 at 21:15 UTC

    It's a tall order to try and reason out that problem purely from the description. Replicating the behaviour outside of your setup would be near impossible. However, there are a few question you could try to answer that may shed some light.

    You say that "I have run this script many times on a few different machines and have found that the problem only occurs in one directory." and "The directory in question is a shared directory".

    Does that mean that it is the same directory on the same machine that is being shared to all of your test machines via the network? If so

    • Is it the only directory you are accessing as a share?
    • Is it the only share you are accessing on that machine?
    • Does the problem go away of you run the program on that machine?

    Essentially, you need to try and isolate what it is that is different about that directory or machine from the other directories. For example; is that machine running a different (version of the) OS? Have you checked the event log (assuming its Win) on that machine? Maybe there is a problem with the browser deamon, or the the filesystem that has errors which are slowing things down.

    You might even look for things like the connection speed of the lan segment to which the shared directory is connected. If the machine in question was using a 4MB NIC or a was connected through a slow or overloaded router it might produce some slow down. A few point to point tests of network speed from the command line to the problem share should determine this.

    Is the machine where the directory in question in the same domain as the test machines? I've seen a situation in the past where accounts in one domain accessing a machine in a different domain were authorised to do so, but the process of authentication required a tortuous negotiation path between several domain controllers each time. Worth a mention.

    A few ideas that spring to mind. As a first pass, I would probably replicate the problem directory to a different machine and redirect the share and see if that effects a difference. It's all about trying to isolate the differences.

    Good luck.

    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "Think for yourself!" - Abigail
    Hooray!
[reply]
      All very good questions:
      Is it the only directory you are accessing as a share?
      Is it the only share you are accessing on that machine?
      Does the problem go away of you run the program on that machine?
      The script is running on the same machine as the files I am checking. The files are located in many different directories (most of them are shared) but again, I am accessing them directly from the machine they reside on.

      I suppose I will copy the files to a FAT volume, remove the share, delete the directory and then recreate the directory and share and then copy the files back in and see what happens. Thanks for your in depth reply.
[reply]

        Another thought crosses my mind. Is it possible that the files in that directory were created (and therefore owned) by people accessing the machine across the LAN or WAN? It strikes me that if they are, then obtaining ownership information could require consultation with the domain controllers where the owners 'live'. If that meant that the local machine was having to send queries to WAN remote PDC's, then that could intrduce quite long delays.

        Note: This is pure speculation.

        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "Think for yourself!" - Abigail
        Hooray!
[reply]
Re: Another Strange Win32::Perms Issue
by Beechbone (Friar) on Oct 28, 2003 at 21:04 UTC
    What happens if you visit this directory with the Explorer? This could just be one of the usual Windows issues---I've got a directory that takes about 8 minutes in the Explorer to load. Yeah, it it's really huge: no files and three subdirs...
    Search, Ask, Know
[reply]
      There are no problems with viewing thedirectory via windows explorer or the command prompt. No one has reported any trouble accessing this directory even over the wan.
[reply]

Back to Seekers of Perl Wisdom