I have been learning Perl, so if this is too simple I am sorry. I needed a way to copy the file outputs of the nmon script I run on AIX machines. I wanted to move them to a single machine where there is plenty of disk for it. This is what I came up with.

#!/usr/bin/perl -w =head1 Author: TechFly Name: nmoncopy.pl Description: Copy all nmon files from the /tmp/nmonout folders on the +machiens listed in the machinelistfilename. This will create a direc +tory on the destination if needed, and will use SCP to copy the files +. Then it will use SSH and delete the files (less two days from runt +ime) from the source. Start date: 7-20-2010 Last updated Date: 7-20-2010 =cut use strict; use warnings; use Net::SCP qw(scp); use Net::SSH qw(ssh); my $scp; my $machinelist; my $machinelistfilename = "./machinelist.txt"; my $server; open($machinelist, "<", $machinelistfilename)||die $!; print("\n"); while($server = <$machinelist>){ chomp($server); if(-d "./$server"){ copydata(); }else{ mkdir($server)||die $!; copydata(); } } sub copydata{ $scp = Net::SCP->new($server, "username"); $scp->get("/tmp/nmonout/*", "/export/nmon/$server/") or die $s +cp->{errstr}; ssh($server, "find /tmp/nmonout/ -mtime +2 |xargs rm"); }

Replies are listed 'Best First'.
Re: File copier
by salva (Canon) on Jul 20, 2010 at 21:49 UTC
    Using Net::OpenSSH::Parallel you can run all the operations in parallel. It is probably going to be much faster:
    use Net::OpenSSH::Parallel; my $pssh = Net::OpenSSH::Parallel->new; my $machinelist; my $machinelistfilename = "./machinelist.txt"; open($machinelist, "<", $machinelistfilename) or die $!; while(<$machinelist>) { chomp; mkdir $_; -d $_ or die $!; $pssh->add_host($_, user => "username"); } $pssh->push('*', scp_get => { glob => 1 }, '/tmp/nmonout/*', "./%HOST% +/); $pssh->push('*', command => "find /tmp/nmonout/ -mtime +2 |xargs rm"); $pssh->run;
Re: File copier
by sierpinski (Chaplain) on Jul 22, 2010 at 04:43 UTC
    I did the same thing with Net::SSH::Expect and Parallel::ForkManager and it worked really well. I have one server connect to all the rest, run commands, capture the output and generate a report based on what it finds.

    If you're interested, I can post some of my code tomorrow. (Can't access it from home at the moment.)

      Please do. I always like to see how other people do the same things. Thanks

        Sorry it took longer than I thought... but here goes... I've omitted non-related portions of code, but you should get the idea:
        ($serverlist is a list of servers, 1 per line)
        my $max_procs = 20; # Create the object that will allow parallel processing my $pm = new Parallel::ForkManager($max_procs); # I used this as a counter to keep track of how many servers # have been connected to vs how many have been attempted, # and used the former as a progress indicator that I've # omitted here. $pm->run_on_finish( sub { my ($pid, $exit_code, $ident) = @_; $connected_server_count += $exit_code; $progress = ($count / $total_server_count) * 100; printf "Progress: %4.1f\%\r",$progress; $count++; } ); foreach my $host (0 .. $#serverlist) { my $pid = $pm->start($serverlist[$host]) and next; # Create ssh object to one server my $ssh = Net::SSH::Expect->new ( host => "$serverlist[$host]", user => "$user", raw_pty => 1, restart_timeout_upon_receive => 1, timeout => 6, ssh_option => " -x", log_file=> "/tmp/$serverlist[$host].log", ); # Validate that we have a successful ssh connection $login_output = $ssh->run_ssh() or die "Could start ssh proces +s, error $!\n"; sleep(1); my $ret; my $rc1 = eval{$ret = $ssh->read_all(12);}; unless($rc1) { open(NOCON,">>$NOCONNECT"); print NOCON "$serverlist[$host] - Could not connect -- + skipping.\n"; close(NOCON); $pm->finish(0); # Couldn't connect to this one, skip +to next server in main loop } my $rc = eval{( $ret =~ />\s*|$\s*\z/) or die "where's the rem +ote prompt?";}; if($rc) { if($ret =~ m/[Pp]assword:/) { # print("Server asking for password, key not ins +talled.\n"); open(NOCON,">>$NOCONNECT"); print NOCON "$serverlist[$host] - Asking for p +assword -- skipping.\n"; close(NOCON); $pm->finish(0); # Couldn't connect to this +one, skip to next server in main loop } } $ssh->exec("stty raw -echo"); # Perform all of your checks here, I'll add one for example # Execute dumpadm check - solaris coredump location my $dumpadm_output = $ssh->send("$dumpadm_cmd"); my $line; while ( defined ($line = $ssh->read_line()) ) { if($line =~ m/dedicated/) { if($line =~ m/zvol/) { # do nothing, uses zfs print("Dumpadm uses zvol -- rootfs is +zfs.\n"); } else { open(DUMP,">>$DUMPADM"); print DUMP "$serverlist[$host] $line \ +n"; close(DUMP); } } } # Close ssh connection to this server $ssh->close(); $pm->finish(1); # 1 is successful, returned 0 for failure to +connect } # End of main while loop for server list # Since multiple processes were spawned asynchronously, must wait unti +l they're all finished # before continuing. $pm->wait_all_children; # won't exit completely until all children are + finished # At this point, I just took all my temporary files and cat'd them tog +ether and sent it in an email to my team
        Since I just copied/pasted chunks of code, I doubt it would work if you tried to run it (even after setting the necessary variables) but you should get the idea of how it's set up. Hope that helps!