Re: Execution hangs on File::Find

I have encountered similar behaviour when iterating over rather large directory tree over network, and I believe this *might* help a bit... Although it is hard to say, if your problem is the same, as mine was.

sub Get_FileList{
    @files=();
    @todel=();
    ####### WORKAROUND START
    my $sloppy = ${^WIN32_SLOPPY_STAT};
    ${^WIN32_SLOPPY_STAT} = 1;
    ####### WORKAROUND END
    find(sub {push @files,$File::Find::name if (-f $File::Find::name a
+nd /\.*$/ and stat($File::Find::name)->mtime > $lastepoch);}, $indir)
+;
    find(sub {push @todel,$File::Find::name if (-d $File::Find::name);
+ }, $indir);     
    $lastepoch = time; #Update last execution time for the next run
    $logger->info("New execution time update: $lastepoch.");
    $proccount = scalar(@files);
    $logger->info("Found $proccount new files since last run.");
    ####### REVERT WORKAROUND
    ${^WIN32_SLOPPY_STAT} = $sloppy;
    ####### REVERT WORKAROUND END
}
[download]

See ${^WIN32_SLOPPY_STAT}

Comment on Re: Execution hangs on File::Find Download Code

Replies are listed 'Best First'.
Re^2: Execution hangs on File::Find by Dallaylaen (Chaplain) on Dec 04, 2017 at 12:00 UTC
++ed the workaround, but I'm wondering whether it's possible to use `local ${^WIN32_SLOPPY_STAT} = 1;` instead of saving/restoring the value manually?	[reply]
Re^3: Execution hangs on File::Find by jahero (Pilgrim) on Dec 04, 2017 at 12:28 UTC
It should be possible (might test it in my code later on).	[reply]
Re^2: Execution hangs on File::Find by colox (Sexton) on Dec 04, 2017 at 12:00 UTC
thanks for the share. i quickly tried, however, it didn't make any diff; still stuck on the top directory without logging or throwing anything =(	[reply]
Re^3: Execution hangs on File::Find by jahero (Pilgrim) on Dec 04, 2017 at 12:36 UTC
Then I guess next step should be testing yout code over smaller directory tree, see if it hangs, and/or where it did spend most of the time. I find it hard to believe that File::Find would freeze, although can hardly backup this claim with concrete evidence, so I would probably profile the run (Devel::NYTProf) on something small first. Good luck, perhaps someone with more knowledge then humble me shall help.	[reply]
Re^4: Execution hangs on File::Find by colox (Sexton) on Dec 04, 2017 at 13:17 UTC
hi, after giving it a little more patience to run & wait, you are indeed correct. it is just "S L O W"... but based on the logging (see below), the "overhead" only occurs on the top folder. as you can see from the logs, it took nearly half an hour for it to start traversing the subfolder. i wonder if any other optimization i can do?.. `04-12-2017 05:00:33:917 DEBUG Reached Z: 04-12-2017 05:09:56:199 DEBUG Reached Z:A6C335 04-12-2017 05:09:57:820 DEBUG Reached Z:A6C335/A6C335.4.12 04-12-2017 05:09:58:001 DEBUG Reached Z:A6C335/A6C335.5.09 04-12-2017 05:09:58:182 DEBUG Reached Z:A6C335/A6C335.5.10` [download]	[reply] [d/l]
Re^5: Execution hangs on File::Find by Eily (Monsignor) on Dec 04, 2017 at 13:34 UTC
Re^5: Execution hangs on File::Find by jahero (Pilgrim) on Dec 04, 2017 at 14:20 UTC
Re^6: Execution hangs on File::Find by colox (Sexton) on Dec 04, 2017 at 19:56 UTC
Some notes below your chosen depth have not been shown here
Re^3: Execution hangs on File::Find by Anonymous Monk on Dec 04, 2017 at 12:28 UTC
What does "dir maped" return from cmd Exe?	[reply]


Keep It Simple, Stupid
	PerlMonks