Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Dearest Monks, I am trying to run a script on every file in a directory. What is the best way to do this?
  • Comment on Run a script on every file in a directory

Replies are listed 'Best First'.
Re: Run a script on every file in a directory
by ikegami (Patriarch) on Jun 07, 2010 at 21:15 UTC

    This is a shell question, but you didn't specify which shell you use. I answered for the two most common shells.

    If your script takes multiple file names:

    # bash script * # cmd perl -e"@ARGV = glob('*'); do 'script' or die $@||$!"

    If you're script takes a single file name:

    # bash for q in * ; do script "$q" ; done # cmd for %q in (*) do script "%q"

      Not quite right. Both bash versions will leave out the "hidden" files with names starting with a ., and they will process every other directory entry, not just ordinary files, but also special files like symlinks, directories, device nodes, pipes. The cmd for loop also skips files with the hidden bit set.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

        More like insufficient information in the question, so I answered the typical case. Should the OP answer me the following questions and specify the target shell, I'll provide.

        • Should directory "." be included? (I assumed no)
        • Should directory ".." be included? (I assumed no)
        • Should other directories be included? (I assumed yes)
        • Should other non-plain files be included? (I assumed yes)
        • Should other hidden files be included? (I assumed no)
        • Should subdirectories be visited recursively? (I assumed no)
Re: Run a script on every file in a directory
by toolic (Bishop) on Jun 07, 2010 at 21:19 UTC
    What is the best way to do this?
    The best way probably depends a lot on what your files look like and what you are going to do to them. Does best mean:
    • Fastest
    • Most platform-independent
    • Least memory usage

    If all you want to do is print out lines of files which match the string 'foo', Unix grep would be a good choice:

    grep foo dir/*
    If you provide more details, you will surely get good advice from our fellow Monks (maybe even me:)
Re: Run a script on every file in a directory
by kejohm (Hermit) on Jun 08, 2010 at 08:31 UTC

    I'm a fan of File::Find for directory traversing, if you prefer using Perl rather than the shell. Here is some code:

    #!perl use strict; use warnings; use File::Find; my $dir = shift; die 'No directory' unless $dir; find( sub { # $File::Find::dir is the current directory name, # $_ is the current filename within that directory # $File::Find::name is the complete pathname to the file. # Process file here... }, $dir ); __END__

    You could use file tests, regex matches, etc. to filter out the files that you want.

Re: Run a script on every file in a directory
by happy.barney (Friar) on Jun 08, 2010 at 09:20 UTC
    perl way (best one) is to use File::Find

    if you prefer bash way, keep in mind that there is a limit max command line length or max arguments count.

    # files only in current directory (also hidden) find . -type f -maxdepth 2 -print0 | xargs -0 script.pl # all files in all subdirectories find . -type f -print0 | xargs -0 script.pl # or run script for every file find . -type f -exec script.pl {} \;
Re: Run a script on every file in a directory
by Anonymous Monk on Jun 08, 2010 at 13:13 UTC
    Oh Wise Monks, Those are all good suggestions, but I forgot to mention two critical details: I am running Perl in Windows via Strawberry Perl (so Perl commands are probably the best choice), and my script outputs a file. I would like the output files to rewrite the original files. Can you show me how to do that?