nancylt723 has asked for the wisdom of the Perl Monks concerning the following question:

I have the following code that works. How can I change this so that on the remote server, all files found recursively in $folder_name are renamed to their name with a .bak extension without knowing the names of the sub_folders?

#!/usr/bin/perl use strict; use warnings; use File::Basename; my @patterns = ("*.*"); my $cmd; my $uid = "something"; my $foldername; foreach $ftype (@ftype) { $cmd = "/usr/bin/ssh -q -l $uid $h 'for f in `ls $foldername/$ftype + 2>/dev/null`; do dos2unix \$f ; done'"; system($cmd); }

Replies are listed 'Best First'.
Re: Rename all files on remote server to *.bak recursively
by Corion (Patriarch) on Jun 05, 2014 at 12:52 UTC

    I would change the approach from your script (sending a shell command loop to the remote side) to a three-step approach:

    1. Get the list of all remote files
    2. Munge the list in Perl
    3. Send a list of mv -i commands to the remote side

    Using the three steps makes it much, much easier to review the list of commands before they are executed.

    The relevant parts of the script would be:

    # Get list of remote files my @remote_files= qx(ssh $uid $h 'find $dirname');
    # Munge filenames to commands: my @commands= map { sprintf "mv -i '%s' '%s.bak'", quotemeta($_), quotemeta($_) } @remote_files;
    # Send list of commands to the remote side: # First, a dry-run instead of actually doing that: my $remote= \*STDOUT; # Use this to actually do the remote execution: #open $remote, "| ssh -q -l $uid $h"; print { $remote } join "\n", @commands;
      Corion ++ !! I did not know about the quotemeta function.
      That is quite a neat function indeed..thanks

      Thanks for the suggestion; however my files were not renamed: My code is now:

      #!/usr/bin/perl my $h="testapp01"; my $dirname="/home/wasbatsrv/tmp/testbak"; my $uid="wasbatsrv"; my @remote_files= qx(ssh $uid\@$h 'find $dirname'); # Munge filenames to commands: my @commands= map { sprintf "mv -i '%s' '%s.bak'", quotemeta($_), quo +temeta($_) } @remote_files; # Send list of commands to the remote side: # First, a dry-run instead of actually doing that: my $remote= \*STDOUT; # Use this to actually do the remote execution: open $remote, "| ssh -q $uid\@$h"; print { $remote } join "\n", @commands;

      But the files are not renamed and this is the output:

      Pseudo-terminal will not be allocated because stdin is not a terminal +. mv: cannot stat `\\/home\\/wasbatsrv\\/tmp\\/testbak\\\n': No such fil +e or directory mv: cannot stat `\\/home\\/wasbatsrv\\/tmp\\/testbak\\/testbakr\\\n': +No such file or directory mv: cannot stat `\\/home\\/wasbatsrv\\/tmp\\/testbak\\/testbakr\\/thre +e\\.properties\\\n': No such file or directory mv: cannot stat `\\/home\\/wasbatsrv\\/tmp\\/testbak\\/testbakr\\/four +\\.properties\\\n': No such file or directory mv: cannot stat `\\/home\\/wasbatsrv\\/tmp\\/testbak\\/two\\.propertie +s\\\n': No such file or directory mv: cannot stat `\\/home\\/wasbatsrv\\/tmp\\/testbak\\/one\\.propertie +s\\\n': No such file or directory

      Here is the directory listing:

      [wasbatsrv@testapp01 testbak]$ ls one.properties testbakr two.properties

        Does the command work when you first print it out and then run it manually in the shell?

        I guess the problem results from me using both, single quotes and quotemeta when constructing the command line.

        Maybe you can make sure that no filename contains a backslash or a single quote. Then you can eliminate both calls to quotemeta and replace them by $_ directly.

        Note that your problem has nothing to do with Perl anymore and is only a matter of constructing the correct shell statement now.

Re: Rename all files on remote server to *.bak recursively
by salva (Canon) on Jun 05, 2014 at 14:30 UTC
    Through SFTP using Net::SFTP::Foreign:
    #!/usr/bin/perl use strict; use warnings; use Net::SFTP::Foreign; use Fcntl ':mode'; my $host = 'localhost'; my $dir = shift @ARGV; my $sftp = Net::SFTP::Foreign->new($host); $sftp->find($dir, ordered => 1, wanted => sub { my (undef, $entry) = @_; if (S_ISREG($entry->{a}->perm)) { my $fn = $entry->{filename}; $sftp->rename("$fn", "$fn.bak"); } 0; }, );

    (the ordered flag is a hack required to force find to read full directories before calling the callbacks, otherwise it would mix readdir and rename operations)

Re: Rename all files on remote server to *.bak recursively
by kschwab (Vicar) on Jun 06, 2014 at 04:57 UTC
    You could leverage find and perl on the remote end.
    $remotehost="somehost.tld"; $remotedir="/some/dir/"; $cmd="/usr/bin/ssh $remotehost find $remotedir -type f -print". '| perl -ne \'chomp;$n=$_.q^.bak^;rename($_,$n)\''; print "cmd is [$cmd]\n"; #uncomment after testing #system($cmd);
        The cmd is missing the "find /whatever -type f -print|" that should be at the front. Guessing you made some modifications. The perl command is looking for a list of files to be piped to it.
Re: Rename all files on remote server to *.bak recursively
by locked_user sundialsvc4 (Abbot) on Jun 05, 2014 at 18:54 UTC

    Just put an appropriate Shell script on the remote system and then tell the remote, via SSH, to execute that script.   It will find the files locally to itself and then rename them appropriately.   Provide the directory-path as an argument to that remote shell script.   The shell script in question can easily consist of a find -r command, piped to a sed command to remove the file-extension, piped to an xargs rm command with placeholders.   The initiating system does not need to be (and should not be) concerned with the exact list of files that may be present on the remote ... the remote knows best.   All that the initiating system needs to do is to ask the remote to do, within its (the remote’s ...) own local context, what it is an ideal position to do.   The local Captain does not need to know anything about the remote’s directory-status in order to give that remote Private an Order (“Sir!   Yes Sir!!”™) to be carried out competently at the remote location.

      Actually, we are talking about over 50 remotes, so this might not be practical. But thanks.
        Then you can run then in parallel combining Net::OpenSSH::Parallel and Net::SFTP::Foreign:
        # untested my @host = ...; my $dir = ...; use Fcntl ':mode'; use Net::OpenSSH::Parallel; use Net::SFTP::Foreign; my $pssh = Net::OpenSSH::Parallel->new; for my $host (@host) { $pssh->add_host($host); } sub sftp_rename { my ($label, $ssh) = @_; my $sftp = $ssh->sftp; $sftp->find($dir, ordered => 1, wanted => sub { my (undef, $entry) = @_; if (S_ISREG($entry->{a}->perm)) { my $fn = $entry->{filename}; $sftp->rename("$fn", "$fn.bak"); } 0; }, ); } $pssh->all(parsub => \&sftp_rename); $pssh->run;

        those 50 remotes can work in parallel, which to me is an argument in favour of having the remotes do the work independently (of course, your local host probably is a multitasking System, too, but ...)