cbtshare has asked for the wisdom of the Perl Monks concerning the following question:

Hello All, I have the script below that is suppose to run against a file with a list of ip addresses, check if they are up or not .If they are up just print, if they are not write to a file after it determines if its empty or not, but I also want that file to have no duplicates, so I have the following, seems like over kill to do a simple task.It runs, but when I get to if the host is not up, that section doesn't even run (I know this because I put dead ip addresses in the list.Can you please assist

#!/usr/bin/perl use warnings; use strict; my $awsGone = "/ansible/awsGone"; my $awsLists = "/ansible/awsLists"; my @ips; open(FILE, '<', $awsLists) || die("Could not open file, $!"); @ips = <FILE>; close FILE; foreach my $addy(@ips) { chomp($addy); #Test if up my $retreval=system qq{nc -w 1 $addy 22 > /dev/null 2>&1}; if($retreval==0) { print "This up -> $addy\n"; } else { ##check if the file is empty, dont bother searching for +duplicates if(-z "$awsGone") { print "Host not up\n"; open(WRITE,'+>>',"$awsGone") || die("canot open $awsGone,$ +!"); print WRITE "$addy\n"; close WRITE; #remove dead ip from file system qq{perl -p -i -e "/$addy/" $awsLists}; } else { ##ip is dead, so open file for writing, then check if d +ead ip is already there open(WR,'+>>',"$awsGone") || die("canot open $awsGone,$!") +; while(my $line = <WR>) { if($line =~ /$addy/) { print "already exists\n"; } else { ##Append dead ip to the file print "Host not up now\n"; print WR "$addy\n"; close WR; system qq{perl -p -i -e "/$addy/" $awsLists}; } } } } }
My test file looks like below, all are good except the 3rd and 4th
ip-10-1-0-152.us-west-2.compute.internal ip-10-1-0-239.us-west-2.compute.internal ip-10-55-55-55.us-west-2.compute.internal ip-10-122-122-122.us-west-2.compute.internal ip-10-1-1-143.us-west-2.compute.internal ip-10-1-1-149.us-west-2.compute.internal ip-10-1-1-150.us-west-2.compute.internal ip-10-1-1-167.us-west-2.compute.internal
my results are
This up -> ip-10-1-0-152.us-west-2.compute.internal This up -> ip-10-1-0-239.us-west-2.compute.internal This up -> ip-10-1-1-143.us-west-2.compute.internal This up -> ip-10-1-1-149.us-west-2.compute.internal This up -> ip-10-1-1-150.us-west-2.compute.internal This up -> ip-10-1-1-167.us-west-2.compute.internal

Replies are listed 'Best First'.
Re: Writing to File based on condition
by Laurent_R (Canon) on Feb 16, 2018 at 23:10 UTC
    Hi cbtshare,

    two comments (assuming I understood what you're doing).

    Don't open you output file for append each time through the loop, open it only once at the beginning and close it at the end. This will be more efficient.

    Remove duplicate IPs from the @ips array before you start. There is no point testing whether the IP is alive if you've already found out, only to remove it afterwards if you've already checked that IP.

Re: Writing to File based on condition
by 1nickt (Canon) on Feb 16, 2018 at 23:37 UTC

    Hi, you are trying to read from the file while you have it open for writing. You should read the current values, then do your processing, then open for writing (if you have anything to write).

    Update: I don't believe you are reading from the file with "+>>", now that I test it:

    $ perl -MPath::Tiny -Mstrict -Mautodie -wE ' my $file = Path::Tiny->tempfile; open my $FH, ">", $file; print $FH "$_\n" for qw/foo bar baz/; close $FH; open my $FH2, "<", $file; print "2: $_" while (<$FH2>); close $FH2; open my $FH3, "+>>", $file; print "3: $_" while (<$FH3>); print $FH3 "I was here"; close $FH3; open my $FH4, "<", $file; print "4: $_" while (<$FH4>); close $FH4; '
    Output:
    2: foo 2: bar 2: baz 4: foo 4: bar 4: baz 4: I was here

    Hope this helps!


    The way forward always starts with a minimal test.
      thank you
Re: Writing to File based on condition
by shmem (Chancellor) on Feb 17, 2018 at 10:42 UTC

    Way too complicated. You have a list of hosts, check them for upness, and you want to gather those that are down and blend them into file with no duplicates, right?

    Then just read the hosts from file, determine upness for each and record that in a hash keyed with the host. After that, read the down-file and add those addresses to the down hash if they are not up. Write the keys into a new file, last do a rename. Something like

    my $hostfile = "/ansible/awsLists"; my $deadfile = "/ansible/awsGone"; my $newdeadfile = $deadfile.'-tmp'; my %down; open my $ifh, '<', $hostfile or die "Can't read '$hostfile: $!\n"; while(my $addy = <$ifh>) { chomp $addy; $down{$addy} = system qq{nc -w 1 $addy 22 > /dev/null 2>&1}; # 0 = + up, not 0 = down } open $ifh, '<', $deadfile or die "Can't read '$deadfile: $!\n"; while(my $addy = <$ifh>) { chomp $addy; $down{$addy} = 1 unless ! $down{$addy}; delete $down{$addy} if exists $down{$addy} and ! $down{$addy}; } # only the down ones remain in %down now open my $ofh, '>', $newdeadfile or die "Can't write '$newdeadfile': $! +\n"; print $ofh $down{$_},"\n" for sort keys %down; close $ofh; rename $newdeadfile, $deadfile;

    Maybe the entries in the host file should be validated to determine their existence, i.e. resolving the names.

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
      Thank you, I have the following which works :
      #!/usr/bin/perl use warnings; use strict; my $awsGone = "/ansible/awsGone"; my $awsLists = "/ansible/awsLists"; #open file for appending open(APPEND,'>>',"$awsGone") || die("canot open $awsGone,$!"); #open AWSlIST file for reading open(READ, '<', $awsLists) || die("Could not open file, $!"); #open AWSGONE file for reading open(READGONE, '<', $awsGone) || die("Could not open file, $!"); my @gone = <READGONE>; my @ips; @ips = <READ>; foreach my $addy(@ips) { chomp($addy); #Test if up my $retreval=system qq{nc -w 1 $addy 22 > /dev/null 2>&1}; ##if filesd are good, leave them alone, but if bad put them in $awsGon +e if($retreval==0) { print "This up -> $addy\n"; } else { #check if file is empty, because after ansible runs it + clears this file if(-z "$awsGone") { print "Host not up\n"; print APPEND "$addy\n"; #remove the dead ip from awsLists system qq{sed -i "/$addy/d" "$awsLists"}; } else { ##file is not empty so check if ip is already + in the file my @match = grep{/$addy/}@gone ; if (@match) { print "already exists\n"; system qq{sed -i "/$addy/d" "$awsLists"}; } else { print "Host not up \n"; print APPEND "$addy\n"; system qq{sed -i "/$addy/d" "$awsLists"}; } } } } close APPEND; close READ; close READGONE;
      Thank you very much, I resolved this in my next reply, let me know if we had the same idea,but for now
      my %down; open my $ifh, '<', $hostfile or die "Can't read '$hostfile: $!\n"; while(my $addy = <$ifh>) { chomp $addy; $down{$addy} = system qq{nc -w 1 $addy 22 > /dev/null 2>&1}; # 0 = + up, not 0 = down }
      For the above could I have said the following to put the dead ips into the hash ? Is there a difference?
      @down=<$ifh>; foreach my $line(@down) { push @down{$addy}, $_ ; } or @down{$addy}=<$ifh>;
      Also I am not understanding your logic on this line, all ips dead and alive are now in keys %down, you are assigning the value of 1 unless the ip is not apart of keys %down?
      $down{$addy} = 1 unless ! $down{$addy}; delete $down{$addy} if exists $down{$addy} and ! $down{$addy}; }
      then lastly, couldn't I just have said below, or you had to go through the hash to get \n formatting?
      print $ifh sort keys %down;
        For the above could I have said the following to put the dead ips into the hash ? Is there a difference?

        Yes. There may be hosts in your down file which at that time were down, but at the current run are up again.

        Also I am not understanding your logic on this line, all ips dead and alive are now in keys %down, you are assigning the value of 1 unless the ip is not apart of keys %down?
        I am assigning the value 1 to the key in %down unless the host is not down, which implies it even exists in the hash %down and its value is not 0. So, hosts which aren't present anymore in the hosts file, but in the down file, are skipped. Then in the next line I delete the host if present in %down if it happens to be up.

        then lastly, couldn't I just have said below, or you had to go through the hash to get \n formatting?

        The below is wrong because it uses $ifh, which is input filehandle here, and because only a bare print implies $_, while print to an explicit filehandle does not. You have to mention $_. To get a newline at the end of each print operation, you have to mention that as well, unless you set $\ (output record separator, see perlvar)

        @list = (a..z); { # ok for STDOUT local $\ = $/; # set output record sep to input record sep for this +block print for @list; } { # wrong open my $fh,'>', 'foo' or die "Oops: foo - $!\n"; local $\ = $/; print $fh for @list; # WRONG. This prints GLOB(0x1945bd8) or such 26 + times } # file 'foo' closed here, since $fh is out of scope { # ok open my $fh,'>', 'bar' or die "Oops: bar - $!\n"; my $fh = select $fh; # $fh now default filehandle local $\ = $/; print for @list; # OK select $fh; # restore STDOUT as default filehandle } # file 'bar' closed here, since $fh is out of scope

        Running the above, you will have a..z printed to STDOUT, each character on a line, a file "foo" with 0 bytes and a file "bar" with 52 bytes.

        perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: Writing to File based on condition
by Anonymous Monk on Feb 16, 2018 at 22:34 UTC
    Replace hour system calls with subroutine calls. Write more subroutines!