Rina has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I know how to open 1 file in order to search and replace a particular string. How do I structure a script to do the same operation in files in all subdirectories? A quick answer will be greatly appreciated. Thanks

Replies are listed 'Best First'.
Re: Search & Replace in subdirectory files
by borisz (Canon) on Apr 23, 2004 at 14:16 UTC
    Use the File::Find module to grab a list of files and then work on that list one by one.
    Boris
Re: Search & Replace in subdirectory files
by Fletch (Bishop) on Apr 23, 2004 at 14:17 UTC
Re: Search & Replace in subdirectory files
by bassplayer (Monsignor) on Apr 23, 2004 at 14:28 UTC
    # perl -p -i.bak -e 's/bilbo/frodo/g' *

    This (at command prompt) will change all instances of 'bilbo' to 'frodo' in all files in the directory, creating a backup called filename.bak.

    Not really sure how to make it recurse subdirectories other than adding /* on the end for each level needed:

    # perl -p -i.bak -e 's/bilbo/frodo/g' */*

    Be careful...

    bassplayer

      Not really sure how to make it recurse subdirectories other than adding /* on the end for each level needed:
       # perl -p -i.bak -e 's/bilbo/frodo/g' */*
      Then we are back at File::Find.
      #!/usr/bin/perl use File::Find; my @dirs = @ARGV; @ARGV = (); File::Find::find( { wanted => sub { push @ARGV, $File::Find::name if -f } }, @dirs ); $^I = '.bak'; local $_; while ( defined( $_ = <ARGV> ) ) { s/bilbo/frodo/g; print; }
      Boris
        Not necessarily. File::Find is the solution I would use if the script is to be run regularly. However, with a one time change, I would still use the command line. It's just so easy. If I have five levels to recurse, I run it five times. Granted, if there is an unknown level of subdiectories, then a folder could be missed, but this one-liner has served me well and I just thought I'd share it. TIMTOWTDI, right? I guess I should have explained that in previous reply.

        bassplayer

      Thank you for your replies so far; I've been reading the file::find references you gave me, going through the book, and testing your suggestions.

      However, I will be more specific because I haven't gotten the result I need yet. There are some 350+ folders & subfolders so I need this to recurse.

      The line where that substring exists looks like this:

      //$Log: \\Server\Dir1\subDir1\subDir2\filename.cpp $

      I want to search every file to replace every instance of "$Log:" with "$History:".

      Either nothing happens, or the entire line before "filename.cpp $" disappears when I do one of the following:

      $InputArray =~ s/Log/History/gi;

      $InputArray =~ s/\$Log/\$History/gi;

      $InputArray =~ s/\$Log\:/\$History\:/gi;
        I can not see, how you lose a line, that is a error somewhere else. To replace '$Log:' with '$History:' this is enough.
        $input =~ s/\$Log:/\$History:/gi;
        Note that $InputArray is _not_ a array.
        Boris
Re: Search & Replace in subdirectory files
by halley (Prior) on Apr 23, 2004 at 16:50 UTC
    % find path/ -name "*.foo" -exec perl -p -i~ -e 's/\$Log:/\$History:/g' {} \;

    --
    [ e d @ h a l l e y . c c ]

      "perl -p -i.bak -e 's/\$Log/\$History/gi' * " at the command line only creates filename.ext.bak, nothing else.

      Ed, I don't understand how to incorporate your suggestion.

      Boris, you're right, but I only showed a snippet of my code. I had something else being sent to the output file instead of $InputArray.

      Below is the full script, which works on a single file. Actually, I don't want to create a new file, I just want to modify the existing file, but this is what I know to do so far. I'm thinking there's a way to use "readdir" to provide the list of files so that the first line would read something like " Open (InputFile, $FileArray)".

      Thanks a LOT!!!



      open( InputFile, "FirstFile.txt" );
      open( OutputFile, ">SecondFile.txt" );
      @InputArrays = <InputFile>;

      foreach $InputArray ( @InputArrays)
      {
      $InputArray =~ s/\$Log/\$History/gi;
      chomp ($InputArray);
      @myColumns = split(/\\/, $InputArray);
      print (OutputFile "\n$InputArray");
      }

      close( InputFile );
      close( OutputFile );
        Hello Rina

        The following code uses File::Find and Tie::File. Note that Tie::File reads each line into an array so you can attempt a substitution. It also edits the file in place so that the original file will be changed to one with the substitutions. Be *careful* of this code as it will change your original files! Maybe set up a dummy directory with some dummy files to test it. I did to test this script! Note that this will change all files in the top directory, /some/dir_name, and goes through each sub_directory looking at all the files.

        #!/usr/bin/perl use strict; use warnings; use Tie::File; use File::Find; my @directories_to_search = ("/some/dir_name"); find(\&wanted, @directories_to_search); sub wanted { if (-f) { tie my @array, 'Tie::File', $_ or die $!; s/\$Log/\$History/gi for @array; } }
        Hope this helps

        Chris

Re: Search & Replace in subdirectory files
by tkil (Monk) on Apr 23, 2004 at 22:55 UTC

    If you're on a unix-like system, you can use plain old find to find all files in this directory or subdirectories, then use perl with "-i" and "-p" to operate on each of them:

    find . -type f \! -name '*.bak' -print0 | \ xargs -0 perl -i.bak -plwe 's/foo/bar/g'

    Modify the arguments to find as appropriate. This is especially nice if you are only performing this substitution on ".html" or ".txt" files, as that removes the need to check for ".bak" files. For example, if you only want to edit files ending with ".html":

    find . -type f -name '*.html' -print0 | \ xargs -0 perl -i.bak -plwe 's/foo/bar/g'

    Also, some older / vendor-supplied versions of find and xargs don't have -print0 and -0; you can just use -print on the find, and remove the -0 from the xargs argument list. (They are there for added security, in case you have files with newlines in their names.)

    If you are on a non-unix system, the File::Find approaches should work. (Or you can instally Cygwin etc.)

    Finally, the details of the perl flags there are described in perlrun; the important one here is "-i.bak", which makes a copy of each file named on the command line then modifies the original in-place (or so you can consider it; more likely, it renames the original to the backup filename, then reads from the backup, operates on the contents, then writes to the original filename).

    So, this perl command:

    perl -i.bak -plwe 's/foo/bar/g' file1.txt file2.txt file3.txt

    Is roughly equivalent to this shell loop:

    for i in file1.txt file2.txt file3.txt do mv $i $i.bak perl -plwe 's/foo/bar/g' < $i.bak > $i done

    (Yes, I know that the input redirection is unnecessary there; I left it in so that it is more obvious that perl is being used as a filter.)

      if your grep does recursion, you can even use it to find the files:
      perl -i.bak -plwe 's/foo/bar/g' `grep -r -l foo .`