pjzero@90 has asked for the wisdom of the Perl Monks concerning the following question:

Question, I'm trying to scp output/file from one machine to another without using Net::SCP, any ideas? Example:
#system("/usr/bin/tmp",$output,"$USER\@$host:/tmp/LOGS/");
While trying to do this in a for loop:
my @OUTPUT = `ls -ltr /tmp` foreach my $output (@OUTPUT) { chomp ($output); system("scp $output user@server:/tmp/LOGS"); } scp: illegal option -- w usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file +] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...] [[user@]host2:]file2 scp: illegal option -- w usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file +] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...] [[user@]host2:]file2 scp: illegal option -- w usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file +] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...] [[user@]host2:]file2 scp: illegal option -- w usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file +] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...] [[user@]host2:]file2 scp: illegal option -- w usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file +] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...] [[user@]host2:]file2 scp: illegal option -- w usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file +] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...] [[user@]host2:]file2 scp: illegal option -- w usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file +] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 [...] [[user@]host2:]file2
No luck..... Let's try this again, I do a ssh over to a host, and grab certain files from /tmp, this works fine, I then want to send these logs to a second host. my @OUTPUT = `ssh $USER\@$host ls -ltr /tmp | grep logs`; foreach my $output (@OUTPUT) { chomp ($output); system("/usr/bin/scp", "--", $output, "user\@host:/home/user/logs"; } I now get "No such file or directory", so your right, I need to put the log itself into a file, any ideas or should I use a different approach? Should I use "find"? Thanks, -- pjzero@90

Replies are listed 'Best First'.
Re: scp output without having to use "Net::SCP" in Perl
by sierpinski (Chaplain) on Jun 02, 2015 at 21:18 UTC
    /usr/bin/tmp

    Should probably try /usr/bin/scp.. ;-)

        Try adding a double dash after the command to stop option processing:
        system("/usr/bin/scp", "--", $output, "$USER\@$host:/tmp/LOGS/");
Re: scp output without having to use "Net::SCP" in Perl
by Laurent_R (Canon) on Jun 03, 2015 at 10:36 UTC
    Hmm, three small things.

    `ls -ltr /tmp` does not give you a list of files, but a list of files with their attributes and characteristics, while you need the bare file names. Also note that you need a semi-colon at the end of that line;

    When you try your scp $output command, you fail to specify the path to $output (/tmp), so that your program is unlikely to find it.

    In this line:

    system("scp $output user@server:/tmp/LOGS");
    you need to escape the @ to prevent Perl from thinking you are trying to manipulate an array called @server. So, something like this:
    system("scp $output user\@server:/tmp/LOGS");
    When trying to do such things, I usually find it convenient to build my full command into a variable and to print that variable to check exactly which command I'll issue to the system. Something like this:
    my $command = "scp $output user\@server:/tmp/LOGS"; print "Issuing this command: $command\n"; system("$command");
Re: scp output without having to use "Net::SCP" in Perl
by stevieb (Canon) on Jun 02, 2015 at 21:59 UTC

    What exactly is in $output? It could be being mistaken for an argument to scp.

    Try hard setting all the system() arguments manually to their actual values, and go from there. For instance, this works for me:

    perl -e '$o="this.fil"; system("/usr/bin/scp", $o, "steve\@hostname:/h +ome/steve/");'

    -stevieb

      stevieb, $output is the output of a command, for example, let's say $output = `ls -ltra /tmp`; Thanks, -- pjzero@90
        Then try something like this:
        system("ls -lrta /tmp | ssh $user\@$host 'cat >/tmp/LOGS/'");

        You can not use scp to copy data from an scalar in the process memory. It only works for files stored in the file system.

        You're trying to scp a command, and the output, to another host? Try breaking that up into separate tasks -- this is what /tmp is for. Connect via ssh to the host and run your command, redirecting the output to /tmp/someuniquefile, then scp the file to whatever location you want. When you start introducing commands with pipes and such in variables for remote execution, the complexity increases. Do one task at a time, and you can manually verify that they work separately.

        Edit: I misread a piece -- you don't scp data, you scp a file. Save the output to /tmp/something, and scp that.
Re: scp output without having to use "Net::SCP" in Perl
by GotToBTru (Prior) on Jun 03, 2015 at 15:10 UTC

    You're getting more than you need from your ls command and it is confusing things.

    $: ls -ltr /tmp total 87120 -rw-r--r-- 1 root system 2206 Jul 23 2012 install_cltp +kg_trace.12255278 -rw-r--r-- 1 root system 2165 Jul 23 2012 install_java +pkg_trace.13893676 -rw-r--r-- 1 root system 2157 Jul 23 2012 install_jrep +kg_trace.13893676 -rw-r--r-- 1 root system 3981 Jul 23 2012 SYMCnbp-init +tab.12255278.i ... $: ls -1tr /tmp install_cltpkg_trace.12255278 install_javapkg_trace.13893676 install_jrepkg_trace.13893676 SYMCnbp-inittab.12255278.i SYMCnbclt-inittab.12255278.r ... $:
    Dum Spiro Spero
Re: scp output without having to use "Net::SCP" in Perl
by lonewolf28 (Beadle) on Jun 04, 2015 at 22:15 UTC

    Hi, The below code works fine.

    chdir '/user/perl_monks' || die "cannot change dir: $!"; open(my $fh, "ls|") || die "cannot open the file handle:$!"; my @files = <$fh>; chomp @files; for ( @files) { system ( "scp $_ username\@host:/any/folder/" ); }

    I used file handle to extract the contents and remove the newline char. I also found that special characters like @ inside the system qq needs to be escaped.

      Hi, The below code works fine.

      No, it does not. It actually is dangerously wrong.

      chdir '/user/perl_monks' || die "cannot change dir: $!"; open(my $fh, "ls|") || die "cannot open the file handle:$!"; my @files = <$fh>; chomp @files; for ( @files) { system ( "scp $_ username\@host:/any/folder/" ); }

      Here is what's wrong with this code:

      1. Newlines are legal in filenames. So, ls may return a single filename splitted over two or more lines, resulting in a filename splitted over two or more elements of @files.
      2. At least GNU ls behaves differently depending on environment variables. It may quote and/or escape special characters in filenames (see GNU ls documentation), but your code assumes sane defaults.
      3. Various characters that have a special meaning to the shell are legal in filenames. (In fact, all characters except ASCII NUL and / are legal in filenames.) So you may legally name a file $(rm -rf /) or $(rm -rf *). Because your code use the single-argument form of system, and it does not clean up $_ before interpolation, the shell will be invoked, and it will interpret those filenames according to shell rules. So, with these filenames, you will likely delete a lot of files.
      4. You blindly assume that system and scp succeed.
      5. Shells are different. Quoting rules change from shell to shell, so there is no safe way to handle the shell.

      And here is how to fix it:

      1. Don't use the shells to read filenames from a directory. glob will do, as will opendir, readdir, closedir, and IO::Dir. For a simple "ls" with sane defaults, glob('*') should do the job.
      2. See above. Avoiding ls also avoids all quoting and escaping issues of ls.
      3. Use the multiple-argument form of system, i.e. system('/path/to/scp',$_,'username@host:/any/folder');.
      4. Check the return value of system to see if system or scp failed. Also look at $!.
      5. Avoid the shells. It just makes life harder. If you want a harder life, read http://www.in-ulm.de/~mascheck/various/.
      I also found that special characters like @ inside the system qq needs to be escaped.

      This has nothing to do with system, you always have to escape @ inside double quoted strings.

      Also note that the qx// operator, also in the backticks form, can also be used in list context. There, it returns one list element per line of output. (This is documented in I/O Operators.) You don't have to use pipe-open and readline (<$handle>), chomp(my @files=`ls`) would be sufficient. But see above for why this is at least problematic, if not dangerous.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)