I'd get rid of the nested sub altogether and take advantage of Perl's array handling to clean up the loop handling. Using early exits as soon as an error is detected cleans up the code and reduces execution time (although that's unlikely to be an issue). Consider:

use strict;
use warnings;

open TXT, "<masks.txt" or die "Can't open masks.txt\n";

while (<TXT>) {
    chomp;
    my @words = split (/ +/);
    print "Checking mask $words[2]\n";
    my $maskvalid = ValidateMasks ($words[2]);
    print "Mask is $maskvalid\n\n";
}

sub ValidateMasks {
    my $mask = shift;
    
    return 0 unless defined $mask;
    my @octets = split /\./, $mask;

    print "\toctets array is @octets\n";

    return 0 unless 4 == @octets; # Bad if too few parts
    
    my $flagvalue = 0;

    for my $octet (@octets) {
        return 0 if $octet > 255;     # bad if value is above 255
        print "\tchecking octet $octet\n";
        next if ($octet eq $flagvalue);    # good if equal to flag val
+ue
        print "\t\tUnequal: oc is $octet, flagvalue is $flagvalue\n";
        
        # bad if this is a second non-zero and it's not equal to 255.
        return 0 if $flagvalue; 
        $flagvalue = 255;    # All further octets after this one must 
+be 255

        # This is the first non-zero octet.
        # Should be equal to a power of 2 minus 1
        # X is a power of 2 if (X & X-1) = 0
        # $octet + 0 forces value to number vice string
        return 0 unless ($octet + 0) & ($octet + 1);
    }
    
    return 1;
}
[download]

Oh, and don't use prototyped subs, they generally don't do what you expect (see Gratuitous use of Perl Prototypes).

I still don't understand why $flagvalue is retaining it's value when control clearly exits and reenters the parent sub....

Lexical variables don't follow the rules of dynamic scoping. They follow the rules of lexical scoping.

If you insist on using nested subs, there are ways & means to implement them e.g. using anonymous subs e.g. (IIRC - since I've not used them since I discovered the wonders of perl)...

sub some_sub {
  my $var;
  my $nested_sub = sub {
    .
    .
    if ($var eq q/value/) {
      # Do something
    } else {
      # Do something else
      $var = q/some_other_val/;
    }
  };

  # Do some stuff
  $var = q/value/;
  &$nested_sub(args);
  if ($val eq q/some_other_val/) {
    # Do some more stuff
  } else {
    # Do something else entirely
  }
  .
}
[download]

The thing is that you cannot have locally scoped subroutines. All named subroutines are implicitly package level things.

Now, you can declare a named subroutine inside another subroutine. That inner subroutine comes into existence without having to execute the outer subroutine. But, because all named subroutines are all at package level, the inner subroutine can be called from outside the outer one.

So, if the inner subroutine refers to lexical variables belonging to the outer subroutine, Perl has a problem. Those variables notionally don't exist until the outer subroutine is entered. It's a mess, which Perl resolves by inventing new versions of the variables apparently shared by the inner and outer subroutines. Those new versions are effectively outside the inner subroutine -- so behave, as you observed, much like 'C' statics.

So:

  sub outer {
    my $foo ;
    ....
    sub inner {
      $foo++ ;
    } ;
  } ;
[download]

  sub outer {
    my $foo ;
    ....
  } ;

  { my $foo ;

    sub inner {
      $foo++ ;
    } ;
  } ;
[download]

(You can declare named subroutines in other blocks, such as inside an eval, which is probably why you can do so in a subroutine. But that's pure speculation.)

Thanks for the lucid explanation of why the problem occurred. I wasn't aware that you could not have locally scoped subroutines. As is probably obvious from my code, I came to Perl by way of C and am still learning the nuances of the language.

There's another discussion which also covers nested named subroutines...

Mkay ... and what would you expect to find in those variables if you call the "inner" subroutine directly, not via the "outer" one?

I didn't expect the inner sub to be visible outside the outer sub. Thanks to the fine Perlmonks here, I've learned the error of my ways.