in reply to Re: Re: Re: Need one-liner to s///g in all sub-dirs
in thread Need one-liner to s///g in all sub-dirs

I have indeed read perlman:perlrun (several times this evening) and I recognize the code you are quoting from that source. I am sure all this seems pretty trivial to folk who use it all the time -- but it is daunting to the uninitiated. And frustrating. I could have writen several scripts to do this task using different and interesting approaches and taken a looong walk in the time I have spent on this trying to get some experience with command line switches. I am presuming upon your patience...

Okay: Encourged by your insistence, I finally found this bit in Camel 3, page 496: The -p loop also contains an implicit select(ARGVOUT); for each file.   perlrun just says, "Note that the lines are printed automatically." It omits to say at that point "to the file in question". It's probably there somewhere. And it's clear once you know it. I just missed it.

All of which brings my to my tired question (revised a little each time). And I ask again because this part is still not clear. Assuming that the files are already backed up, does not this simple line: %perl -pie 's/old/new/g' ./*/* do everything I need without the use of find.... When I am doing things on unfamiliar turf, I like (if possible) to keep it simple and clear.

Update: Still reading... perlrun does contain the explanation of the implicit select(ARGVOUT);. But, as in Camel 3 (which simply reprints the whole section) it is under the -i, not -p. IMHO, the briefer code given under -p is misleading since it claims to explain what -p does but fails to mention the implicit 'select'. This is a crucial detail and the reason I deliberately chose not to use it at first. I had no need to print to STDOUT (which is what the code under -p, as given, would do).

Replies are listed 'Best First'.
Re: Re: Re: Re: Re: Need one-liner to s///g in all sub-dirs
by blakem (Monsignor) on Aug 25, 2001 at 09:32 UTC
    your command will only do the replacement on files that are exactly two levels below your working directory. Perhaps that is what you want, perhaps not. To list the files that your line would affect, use:

    ls ./*/*
    Update: we need the -d flag to avoid getting the listing of directories... sorry bout that.

    ls -d ./*/*
    To list the files the 'find' solutions would affect, use:
    find . -type f -print

    -Blake

      I have done as you suggested (did you?). And my observation is that ls ./*/* reports first on files that are one level below the current directory, i.e. files in the immediate sub-directories (which is what I want). And in addition ls seems to automatically recurse if given a wildcard. So it also reports recursively on sub-directories of those one-level-down directories. My explanation:
      . the current directory /* all dirs in next level down /* all files in those dirs
      and recurse from there.

      If the recursion happens with the perl example, it doesn't matter in my case since there is nothing there.

      I'd still like to know if %perl -pie 's/old/new/g' ./*/* does what I want.

      Meditation: This whole thing has taken a direction that I am bemused by. I thought I was asking a simple and straightforward question. The discussion has ranged far and wide. And I find myself in the position of 'arguing' with answers that seem not to answer the question I asked but rather go in different directions. If I am seeming ungrateful, ungracious, or argumentative, I apologize. I really am just trying to get clarity on my measily half-line solution.

      If my solution does not do what I want, just say that and explain where I have gone astray. And if my solution fails, I would be very happy to see a pure-perl command-line solution if that is possible.

      Thanks to all for your (presumed) patience.

        Lets break the '.*/*' approach down a bit, to help you understand the difference between the two

        perl -pie 's/old/new/g' ./*/*
        The first thing that happens when you enter this command into your shell, is that the shell expands the wildcards. This is important.... perl doesn't ever see the './*/*' construct. The shell then executes the expanded command which now looks like:
        perl -pie 's/old/new/g' subdir/file1 subdir/file2 subdir2/file1 subdir +2/subsubdir/
        And perl does its magic inplace editing on this list of files. (note, see the update to my previous post) No recursion is done. You've simply said, execute this perl command on the following files. Those files being ones that the shell matched with its './*/*' pattern.

        In otherwords, you could use that command like so:

        perl -pi -e 's/old/new/g' file1 dir1/file2 /usr/local/file3 /tmp/*
        You've passed the perl command a set of files which it will work with. (thats what the empty <> operator does that you've seen in the perlrun docs)

        On the other hand, the find solution will walk the filesystem executing the perl command on each file that matches its criteria (here its just </code>-type f</code> which means a regular file) Depending on your filesystem layout, this can be much more powerful than the other solution.

        Said another way... the find solution is equivalent to the following series of commands:

        perl -pie 's/old/new/g' ./* perl -pie 's/old/new/g' ./*/* perl -pie 's/old/new/g' ./*/*/* perl -pie 's/old/new/g' ./*/*/*/* perl -pie 's/old/new/g' ./*/*/*/*/* etc.....
        Of course you'll get errors about executing the command on directories, and you'll soon hit your shells 'too many arguments' limit. Therefore find is a better general solution, but './*/*' might work fine in any one instance.

        -Blake

Re: Re: Re: Re: Re: Need one-liner to s///g in all sub-dirs
by rob_au (Abbot) on Aug 25, 2001 at 09:47 UTC
    All of which brings my to my tired question (revised a little each time). And I ask again because this part is still not clear. Assuming that the files are already backed up, does not this simple line:
     
    %perl -pie 's/old/new/g' ./*/*

     
    It comes close ... what this line does not do is act on the files of the sub-directories of your sub-directories. Wih your original method, files in the current directory and its immediate subdirectories are being acted upon, but no further. No files in subdirectories therein would be acted upon, negating the 'all sub-directories' requirement of your one-liner. Depending on your directory structure, this could leave a large number of files untouched.
     
    The best way to describe this is through example. From the /usr directory of my mail server:
     
    budapest:/usr# ls -l ./*/* | wc -l 4505 budapest:/usr# find . -type f -print | wc -l 22838

     
    The difference in number of files is quite apparent - If however, you have no further subdirectories beneath those of your current directory, then there should be no problems, but if there is, potentially a large number of files could be missed.
     

     
    Ooohhh, Rob no beer function well without!
      When I originally said, "all files in all subdirectories of the current directory" I can see now that that was ambiguous and was no doubt the cause of some of the confusion. I should have said "all the files in the immediate subdirectories of the current directory."

      With that clarification, I think I understand you that the latest version of my proposal will do the job just fine.

      ...Thank you.

Re: Re(4): Need one-liner to s///g in all sub-dirs
by chipmunk (Parson) on Aug 25, 2001 at 16:47 UTC
    The documentation of -p omits to say that the lines are printed "to the file in question" because -p does not turn on that behavior. -p creates a loop which reads in a line into $_ with <>, executes your code, and prints $_. It prints $_ to the currently selected filehandle, which is STDOUT by default.

    It's the -i switch that enables the "...to the file in question" behavior. The documentation explains that -i "specifies that files processed by the <> construct are to be edited in-place. It does this by renaming the input file, opening the output file by the original name, and selecting that output file as the default for print() statements."

    Note that the -i command line switch actually takes an optional argument; an extension that is used to make a backup copy of the original file. For example, -i~ would create a backup file with a tilde at the end of the name, a la emacs.

    Since -i takes an argument, perl -pie 's/old/new/g' ./*/* treats the letter e as the backup extension, and tries to open a script named 's/old/new/g'. The -i and -e switches need to be separated: perl -pi -e 's/old/new/g' ./*/*