Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^3: Execution hangs on File::Find

by jahero (Pilgrim)
on Dec 04, 2017 at 12:36 UTC ( [id://1204852]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Execution hangs on File::Find
in thread Execution hangs on File::Find

Then I guess next step should be testing yout code over smaller directory tree, see if it hangs, and/or where it did spend most of the time.

I find it hard to believe that File::Find would freeze, although can hardly backup this claim with concrete evidence, so I would probably profile the run (Devel::NYTProf) on something small first.

Good luck, perhaps someone with more knowledge then humble me shall help.

Replies are listed 'Best First'.
Re^4: Execution hangs on File::Find
by colox (Sexton) on Dec 04, 2017 at 13:17 UTC

    hi, after giving it a little more patience to run & wait, you are indeed correct. it is just "S L O W"... but based on the logging (see below), the "overhead" only occurs on the top folder. as you can see from the logs, it took nearly half an hour for it to start traversing the subfolder. i wonder if any other optimization i can do?..

    04-12-2017 05:00:33:917 DEBUG Reached Z: 04-12-2017 05:09:56:199 DEBUG Reached Z:A6C335 04-12-2017 05:09:57:820 DEBUG Reached Z:A6C335/A6C335.4.12 04-12-2017 05:09:58:001 DEBUG Reached Z:A6C335/A6C335.5.09 04-12-2017 05:09:58:182 DEBUG Reached Z:A6C335/A6C335.5.10

      Try "Z:/" rather than "Z:", without the / it searches at the current position on Z: (eg, if you cd to "Z:/A6C335" and search on "Z:", you'll already be inside A6C335). I'm not sure if that can be the problem, but at the very least it will be less confusing. If it's still not better, try adding more logs to find precisely which step is taking time.

      ... and this is exactly why I am setting the ${^WIN32_SLOPPY_STAT} in my code. It leads to not openning the file on stat, which can lead to rather significant performance gain.

        hi, but im also testing if the file timestamp is newer than the last execution time. removing stat would not serve that purpose right?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1204852]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (2)
As of 2024-04-26 05:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found