Category: Networking Code
Author/Contact Info Bill Farley/bill.farley@gmail.com
Description: Analyzes tab-delimited file with network blocks for duplicates and subnet overlaps. Created for use with CISCO SESM, but could be used with any application that uses stored network blocks.
#!/usr/bin/perl

#This program is free software: you can redistribute it and/or modify 
+it under the terms
#of the GNU General Public License as published by the Free Software F
+oundation, either 
#version 3 of the license, or any later version.

#This program is distibuted in the hope that it will be useful, but WI
+THOUT ANY WARRANTY;
#without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+ PARTICULAR PURPOSE.
#See the GNU General Public License for more details.

#You should have received a copy of the GNU General Public License alo
+ng with this program.
#If not, see <http://www.gnu.org/licenses/>

#To contact the author of this software, send an e-mail to Bill Farley
+ (bill.farley@gmail.com).

#This script is designed to recognize and identify duplicate network b
+locks and/or overlapping
#subnets from a tab delimited file.  It requires that your tab delimit
+ed file be in the following format:
#example
#1.1.1.1    255.255.0.0
#2.2.2.2    255.255.255.0
#
#When finished, save your tab-delimited file to the same directory as 
+this script

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#

system(clear);        #clear user's terminal screen

print "Starting Script........\n\n";    #notify user that script has s
+tarted

$start = localtime;    #timestamp of when script started
$found = 0;    #counter to pass information back to the user after scr
+ipt is complete.

#open tab-delimited file and create an array for each line
open(FILE, "<$filename") || die ("Error 103 - Could not open $filename
+\n");
    $line_counter = 0; #counts the number of lines in the file
    while (<FILE>) {
        chomp $_;
        #split the line into array variables for Network and Subnetmas
+k
        ($network[$line_counter],$subnetmask[$line_counter]) = split(/
+\t/,$_);
        $line_counter++;
    }

close(FILE);

&duplicates;    #check the network array for duplicate network blocks
print "\n";    #prints a blank line between duplicates and overlaps wh
+en displayed back to user

#decriment subnetmask by one bit
$snc = 0;
foreach $network(@network) {
    $original_network = $network;    #new variable will be used as a r
+eference if any overlapping networks are found
    &slashnot;    #calls subroutine to convert decimal notation for su
+bnetmask to slash notation
    $snc++;
}
#provide user with status of script
if($found gt 0) {
    print "\nScript Complete........  Please review the previous warni
+ng(s) listed above.\n\n";
    }
else {
    print "\nScript Complete........  No duplicates or overlaps were f
+ound.\n\n";
}

$stop = localtime;

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#

sub BEGIN {

system(clear);    #clears the user's terminal

print "Subnetoverlap.pl Copyright (C) 2005, 2007 Bill Farley\n\n";
print "This program is free software: you can redistribute it and/or m
+odify it under the terms\n";
print "of the GNU General Public License as published by the Free Soft
+ware Foundation, either\n";
print "version 3 of the license, or any later version.\n";
print "This program comes with ABSOLUTELY NO WARRANTY.\n\n\n";

print "What is the filename of the tab-delimited file? (Example: file.
+txt): "; $filename = <STDIN>;

#open user provided file and remove all extra hidden characters then r
+ewrite to a new file so script will execute properly
$tempfilename = ("$filename" . "temp.txt");
open(TEXT,"<$filename") || die ("Error 101 - Can't open $filename");  
+      #user's file
open(NEW, ">$tempfilename") || die ("Error 102 - Can't open $tempfilen
+ame");        #new file that script will use
$text_counter = 0;
while (<TEXT>) {
        chomp $_;
        $_ =~ s/\r\n//g;    #checking for extra hidden characters in u
+ser's file and variables
        $_ =~s/\r//g;        #checking for extra hidden characters in 
+user's file and variables
        $_ =~s/\n//g;        #checking for extra hidden characters in 
+user's file and variables
        (($network[$text_counter], ($subnetmask[$text_counter])) = spl
+it /\t/,$_);    #separating line into two arrays
        chomp $network[$text_counter]; chomp $subnetmask[$text_counter
+];    #removing any newline characters that may be in variables
        $text_counter++;
}
                                                                      
+                                                                     
+                  
close(TEXT);
#write array variables to new file
$text_counter2 = 0;
until ($text_counter2 eq $text_counter) {
        print NEW "$network[$text_counter2]\t$subnetmask[$text_counter
+2]\n";    #writing variables into new file in tab-delimited format
        $text_counter2++;
}
                                                                      
+                                                                     
+                  
close(NEW);

#create a hash that corrolates decimal notation to slash notation
        %slash = ("128.0.0.0", 1,"192.0.0.0", 2,"224.0.0.0", 3,"240.0.
+0.0", 4,"248.0.0.0", 5,"252.0.0.0", 6,"254.0.0.0", 7,"255.0.0.0", 8,"
+255.128.0.0", 9,"255.192.0.0", 10,"255.224.0.0", 11,"255.240.0.0", 12
+,"255.248.0.0", 13,"255.252.0.0", 14,"255.254.0.0", 15,"255.255.0.0",
+ 16,"255.255.128.0", 17,"255.255.192.0", 18,"255.255.224.0", 19,"255.
+255.240.0", 20,"255.255.248.0", 21,"255.255.252.0", 22,"255.255.254.0
+", 23,"255.255.255.0", 24,"255.255.255.128", 25,"255.255.255.192", 26
+,"255.255.255.224", 27,"255.255.255.240", 28,"255.255.255.248", 29,"2
+55.255.255.252", 30,"255.255.255.254", 31,"255.255.255.255", 32,); 

#create a hash that corrolates slash notation to decimal notation
    %decimal = (1, "128.0.0.0",2, "192.0.0.0",3, "224.0.0.0",4, "240.0
+.0.0",5, "248.0.0.0",6, "252.0.0.0",7, "254.0.0.0",8, "255.0.0.0",9, 
+"255.128.0.0",10, "255.192.0.0",11, "255.224.0.0",12, "255.240.0.0",1
+3, "255.248.0.0",14, "255.252.0.0",15, "255.254.0.0",16, "255.255.0.0
+",17, "255.255.128.0",18, "255.255.192.0",19, "255.255.224.0",20, "25
+5.255.240.0",21, "255.255.248.0",22, "255.255.252.0",23, "255.255.254
+.0",24, "255.255.255.0",25, "255.255.255.128",26, "255.255.255.192",2
+7, "255.255.255.224",28, "255.255.255.240",29, "255.255.255.248",30, 
+"255.255.255.252",31, "255.255.255.254",32, "255.255.255.255",);
}

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#
sub duplicates {
    $avc = 0;       #This is a counter for the array variables
    foreach $network(@network) {
            while ($avc < $line_counter) {
                    if ($network eq $network[$avc]) {       #matching 
+each array variable against all other variables in the array.
                            $match++;       #counter to keep track of 
+the variable matches since the matching variable is from
                                    #from the array, there will always
+ be at least one match against itself.
                                    #Looking for anything higher than 
+a single match to identify a true duplication
                            if ($match gt 1) {
                                    print "WARNING!!  There is a dupli
+cate network match for $network.\n";
                    $found++;
                            }
                    }
            $avc++;
            }
     $avc = 0;
    $match = 0;
    }
}

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#
sub slashnot {
    foreach $key (keys(%slash)) {
        if  ($key eq $subnetmask[$snc]) {
                $new_slash = ($slash{$subnetmask[$snc]} - 1);    #decr
+imenting subnetmask of original subnetmask by one bit
            &decimalnot;    #calling subroutine to change new slash no
+tation back to decimal notation
        }
    }
}

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#
sub decimalnot {
    foreach $key (keys(%decimal)) {
        if ($key eq $new_slash) {
            $new_subnetmask = $decimal{$key};    #converting slash not
+ation to decimal notation
            &binaryand;    #calling subroutine to identify new network
+ block address
        }
    }
}

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#
sub binaryand {
       ($ipoctet1,$ipoctet2,$ipoctet3,$ipoctet4) = split(/\./,$network
+);       #split up the network block and assign octet variables
       ($snmoctet1,$snmoctet2,$snmoctet3,$snmoctet4) = split(/\./,$new
+_subnetmask);  #split up the subnetmask and assign octet variables
                                                                      
+                                                                     
+                  
        #Due to a bug in Perl, after the split, the new variables need
+ to have an mathmatical operation
        #performed in order to make them numeric variables rather than
+ string variables.
        $num_con1 = $snmoctet1 + $ipoctet1;
        $num_con2 = $snmoctet2 + $ipoctet2;
        $num_con3 = $snmoctet3 + $ipoctet3;
        $num_con4 = $snmoctet4 + $ipoctet4;
                                                                      
+                                                                     
+                  
        #identify the network block address by "AND"ing the ip address
+ and the subnet mask.
        $nwoctet1 = $snmoctet1 & $ipoctet1;
        $nwoctet2 = $snmoctet2 & $ipoctet2;
        $nwoctet3 = $snmoctet3 & $ipoctet3;
        $nwoctet4 = $snmoctet4 & $ipoctet4;

        $new_network = ("$nwoctet1.$nwoctet2.$nwoctet3.$nwoctet4");   
+ #joining octets together into a single variable
        $new_subnetmask = ("$snmoctet1.$snmoctet2.$snmoctet3.$snmoctet
+4");    #joining octets together into a single variable
    &overlap;    #calling subroutine to identfy any overlappping netwo
+rks
}

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#
sub overlap {
    $overlap_counter = 0;
    while($overlap_counter < $line_counter) {
        #match new subnet against all other variables in the array
        if (($new_network eq $network[$overlap_counter]) and ($new_sub
+netmask eq $subnetmask[$overlap_counter])) {
            print "WARNING!!  There is an overlap between $new_network
+ and $original_network\n";
            $found++;
        }
    $overlap_counter++;
        }
    #continue to decriment subnetmask by one bit until subnetmask is a
+ /1
    if ($new_slash ne 1) {
        $new_slash--;    #decrimenting subnetmask slash notation by on
+e bit
        &decimalnot;    #looping to subroutine for conversion back to 
+decimal notation
    }
}
Replies are listed 'Best First'.
Re: Subnet Overlap (fixes)
by ikegami (Patriarch) on Aug 21, 2007 at 22:48 UTC

    Here's a solution that not only addresses all the fixes, but uses a much better algorithm. (It's speed is bound by sort instead of O(N2).)

    #!/usr/bin/perl # # The name of the file containing the data is specified as an argument +. # Alternatively, the data may passed via STDIN. # The data consist of mulitple records, one per line. # Each record has two columns: a network address and its mask. # The columns are seperated by whitespace. # Lines containing only whitespace are ignored. # Comments (anything after a "#") are ignored. # # # Example # ------- # # This is a sample data file. # # 10.0.0.0 255.255.255.0 # Overlaps with 4, 5 & 6 # 10.0.0.0 255.255.255.128 # Overlaps with 5 & 6 # 10.0.0.0 255.255.255.192 # 10.0.0.64 255.255.255.192 # # 11.0.0.0 255.255.255.0 # 11.0.1.0 255.255.255.0 # 11.0.2.0 255.255.254.0 # Overlaps with 11 # 11.0.3.0 255.255.255.0 # 11.0.4.0 255.255.255.0 # # 12.0.0.0 255.255.255.0 # Overlaps with 15 # 12.0.0.1 255.255.255.0 # use strict; use warnings; sub packip4 { return unpack('N', (pack 'C4', split(/\./, shift))); } sub unpackip4 { return join('.', unpack('C4', pack('N', shift))); } sub format_rec { my $rec = @_ ? $_[0] : $_; return sprintf('%s/%s (line %d)', unpackip4($rec->[0]), unpackip4($rec->[1]), $rec->[2], ); } sub overlap { my ($overlaps, $r1, $r2) = @_; if ($r1->[2] > $r2->[2]) { ($r1, $r2) = ($r2, $r1) } push @$overlaps, [ $r1, $r2 ]; } { my @recs; { while (<>) { s/#.*//; s/\s+\z//; next if !length; my ($net, $mask) = split /\s+/; $net = packip4($net); $mask = packip4($mask); my $first = $net & $mask; my $last = $net | ~$mask; push @recs, [ $first, $last, $. ]; } } my @overlaps; if (@recs) { @recs = sort { $a->[0] cmp $b->[0] || $a->[1] cmp $b->[1] } @rec +s; my $crushed = shift @recs; my @components = ( $crushed ); while (@recs) { my $rec = shift @recs; if ($rec->[0] gt $crushed->[1]) { $crushed = $rec; @components = ( $rec ); next; } foreach my $component (@components) { next if $component->[0] gt $rec->[1] || $component->[1] lt $rec->[0]; overlap(\@overlaps, $rec, $component); } $crushed = [ $crushed->[0], ( $crushed->[1] gt $rec->[1] ? $crushed->[1] : $rec->[1] ) ]; push @components, $rec; } } print map { sprintf("%s overlaps %s\n", map format_rec, @$_) } sort { $a->[0][2] <=> $b->[0][2] || $a->[1][2] <=> $b->[1][2] + } @overlaps; }
      Thanks for submitting your fixes. I don't quite understand all the code, so it will give me something to review and learn. Thanks again. -B
        The key tidbit is that the addresses are stored as a 4-byte string, which is why it's appropriate (and required) to use gt instead of > (etc).
Re: Subnet Overlap
by grinder (Bishop) on Aug 21, 2007 at 20:10 UTC

    Hello, I've reviewed your code and I have a number of comments to make. First things first: I hate to break it to you, but a lot of the code could be replaced by a couple of lines of code from a CPAN module, like Net::CIDR or NetAddr::Ip. I'd also wager a beer that there's a bug or two present in your code. That is not to mean you're not a good programmer, but IP address arithmetic is notoriously hard to get right. By using either of these modules, you're getting code that has been thoroughly exercised by people in many contexts, over many years.

    Apart from the design design to clear the screen when the program is run (something that many people find irritating), the execution of the program 'clear' only happens to work because you're not using strict. Thus, the perl interpreter first tries to see if clear is an internal routine, and after having given up, it ends up interpreting the bare word as a string. So it works, but it's a dodgy practice.

    One of the reasons I think that the code may have bugs is because you are using string comparisons ($match gt 1) rather than numeric comparisons ($match > 1). Comparing against one may be safe, but remember that 9 > 10 evaluates to true. Say what you mean, to be certain.

    Elsewhere in the code, a comment says:

    Due to a bug in Perl, after the split, the new variables need to have an mathmatical operation performed in order to make them numeric variables rather than string variables.

    This is quite untrue and more indicative of your own misunderstanding of how Perl deals with strings. If you put a variable in an arithmetic expression, Perl will treat it as a number. Thus "2" + "2" equals 4, but "2" . "2" equals "22". split will return just what you want. Numbers or strings, it's your call.

    Briefly in passing, consider using the 3-arg form of open, don't call functions with ampersand (&func;) and do use strict.

    Also, rather than dealing with the mechanics of opening and closing files, you can ditch a lot of code by just reading from STDIN and writing to STDOUT, and deal with what file is to be read, and to what file the results should be written, from the shell. Less code, less things to go wrong.

    I must apologise if this all comes out sounding a bit harsh, that is not my intent, but I don't have the time to be more diplomatic. We are all here to learn, and if you learn from this post, then we're one step ahead.

    Keep perling!

    • another intruder with the mooring in the heart of the Perl

      This is quite untrue and more indicative of your own misunderstanding of how Perl deals with strings

      Actually, the OP was correct. split does return strings, and he does need to force them into numbers before using & on them.

      my $i = my $x = (split(/\./, '205.0.0.0'))[0]; my $j = my $y = (split(/\./, '123.0.0.0'))[0]; my $ij = $i & $j; $x += 0; $y += 0; my $xy = $x & $y; print("$i & $j = $ij\n"); # 205 & 123 = 001 print("$x & $y = $xy\n"); # 205 & 123 = 73

      Your example shows that +, expecting a number, will convert the string to a number, but the OP is using & which is acts differently for strings and numbers.

      The OP was wrong in calling it a bug, though.

        Thanks for clarifying that for me. Yeah, I should have used a word other than "bug" as it's not truly a bug. Thanks for your feedback. -B
      Thanks for your input. I haven't been using Perl for very long, so I am sure that there are several other ways to do what I did. My first objective was just to see if I could solve a problem. Although using a CPAN module may have been easier and already tested, it doesn't really give me the opportunity to solve the problem on my own. I never really thought too much about the clear screen lines, but I will keep that in mind for any future code I post. I'll be sure to start using strict. As you mentioned, it's just good practice. I am correct in my statement about what is required after using split, however, I should have used better terminology in my comment, and not described it as a bug. I'm not familiar with the 3-arg form of open, so that is something I will look into. Good point on the STDIN and STDOUT. I will keep this in mind for any future code. Thanks again for the feedback, It will definitely give me some things to research and think about for any additional code I write. -B
Re: Subnet Overlap
by ikegami (Patriarch) on Aug 21, 2007 at 20:41 UTC

    grinder already mentioned a number of issues, but there are more:

    • Use of numbered scalars instead of arrays makes for very repetitive code.
    • Speaking of repetition, why are %slash and %decimal built by hand?!?
    • None of your variables are scoped. Adding use strict; will show those errors.
    • The code in the BEGIN block doesn't belong in a BEGIN block.
    • Meet join, the converse of split.
    • Meet $., line counter.
    • The use of clear is not only unnecessary, it's not portable.
    • $_ =~ s/.../.../g; can be written as s/.../.../g;.