in reply to Optimizing files' moving with File::Find & File::Copy modules

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. (Donald Knuth)
Though this statement is not applicable in all situations it certainly is in this one.

Why do you think that "optimiz<ing> it by having _as many_ files as possible in the 'wanted' function would affect the performance? The only thing here you could call (minimal) "overhead" is the repeated calls to "wanted", but that's just the price you pay for using File::Find.

Why do you think that the OS would "move(...) the files all together in _one shot_ or at least in big chunks"? It moves them file by file.

Update: If you perceive performance differences between your perl version and a shell based equivalent, that's because File::Copy moves a file by
  1. opening the target for writing
  2. reading from source and write to the target
  3. unlinking/deleting the source
Depending on OS and filesystem this is less then optimal but also not OS dependent.

If that's no concern you could push the files found by "wanted()" into an array (or better yet use File::Find::Rule, which doesnt have this clunky callback interface) and feed that to the systems "mv", using proper quoting.


holli

You can lead your users to water, but alas, you cannot drown them.

Replies are listed 'Best First'.
Re^2: Optimizing files' moving with File::Find & File::Copy modules
by JavaFan (Canon) on Nov 17, 2009 at 12:28 UTC
    My guess is that the OP wants to cut down on the number of processes - which many monks will tell you is "bad".

    Personally, if I were to move within the file system, I would call rename (and use an external find to find the files); if I were to move from one filesystem to the other, I would realize the entire process is likely to be disk I/O bound, so I wouldn't care about the process calls.

      What processes? Neither File::Find nor File::Copy spawn any external processes.


      holli

      You can lead your users to water, but alas, you cannot drown them.