strat.minger has asked for the wisdom of the Perl Monks concerning the following question:

I have a list of IP Addresses and the amount of data that they have transfered over a period of time, it is in the form:
10.0.0.1 555555 10.2.1.6 444444 10.6.2.8 333333 10.2.1.2 222222 etc...
I am performing some actions on these IP's but ideally I would like to do it by subnet. All the IP Addresses on the network belong to a /29 subnet. I am using NetAddr::IP to then get the network address that corresponds to each individual IP. The list could look like this
10.0.0.0/29 10.0.0.1 555555 10.2.1.0/29 10.2.1.6 444444 10.6.2.0/29 10.6.2.8 333333 10.2.1.0/29 10.2.1.2 222222 etc...
I now want to group together all the lines that have the same network address and add up the bytes transfered. This in theory would like something like:
10.2.1.0/29 666666 10.0.0.0/29 555555 10.6.2.0/29 333333
I then need to run these through some more processes that I have working on the individual IP Addresses at the moment. The problem is that there are a very large amount of IP's and subnets, and they can change over every measurement period as can the data transfered.

At the moment I am trying to use arrays and foreach loops etc.. to try and get this working.

Any help as I struggle on with this would be appreciated.

Replies are listed 'Best First'.
Re: Sorting IP Subnets together
by ikegami (Patriarch) on Apr 12, 2007 at 16:21 UTC

    Remember, an IP address is simply a 32-bit number. The dotted form is just a representation of that number. While the dotted form might be easier to transcribe, it's easier to work with the 32-bit number directly when working with subnets.

    The following actually uses the packed form of the 32-bit numbers. It's easier to obtain, and allows for easy sorting (sort vs sort { $a <=> $b }).

    use strict; use warnings; my $subnet_size = 29; my $subnet_mask = pack('B*', ('1' x $subnet_size) . ('0' x (32-$subnet_size))); my @ip_xfers = ( [ '10.0.0.1', 555555 ], [ '10.2.1.6', 444444 ], [ '10.6.2.8', 333333 ], [ '10.2.1.2', 222222 ], ); my %subnet_xfers; foreach (@ip_xfers) { my ($ip, $xfer) = @$_; my $packed_ip = pack('C4', split(/\./, $ip)); my $subnet = $packed_ip & $subnet_mask; $subnet_xfers{$subnet} ||= 0; $subnet_xfers{$subnet} += $xfer; } foreach my $subnet (sort keys %subnet_xfers) { my $subnet_ip = join('.', unpack('C4', $subnet)); my $xfer = $subnet_xfers{$subnet}; print("$subnet_ip/$subnet_size: $xfer\n"); }
    10.0.0.0/29: 555555 10.2.1.0/29: 666666 10.6.2.8/29: 333333

      Since the OP was using NetAddr::IP, what follows is a version that uses that module.

      I used the same approach as in my earlier post. The main difference is that sorting is trickier because we are working with the string form. A plain (lexical) sort won't do because 20.0.0.0 would sort after 198.162.0.0.

      use strict; use warnings; use NetAddr::IP qw( ); my $subnet_mask = 29; my @ip_xfers = ( [ '10.0.0.1', 555555 ], [ '10.2.1.6', 444444 ], [ '10.6.2.8', 333333 ], [ '10.2.1.2', 222222 ], ); my %subnet_xfers; foreach (@ip_xfers) { my ($ip, $xfer) = @$_; my $subnet = NetAddr::IP->new($ip, $subnet_mask)->network(); my $subnet_ip = $subnet->addr(); # Save NetAddr::IP for sorting. $subnet_xfers{$subnet_ip} ||= [ $subnet, 0 ]; $subnet_xfers{$subnet_ip}[1] += $xfer; } my @sorted_keys = sort { $subnet_xfers{$a}[0]->aton() cmp $subnet_xfers{$b}[0]->aton() } keys %subnet_xfers; foreach (@sorted_keys) { my ($subnet, $xfer) = @{$subnet_xfers{$_}}; print("$subnet: $xfer\n"); }
      10.0.0.0/29: 555555 10.2.1.0/29: 666666 10.6.2.8/29: 333333

      Update: I originally didn't sort the output due to the extra complexity.

Re: Sorting IP Subnets together
by jettero (Monsignor) on Apr 12, 2007 at 15:33 UTC
    I suggest looking at Net::Netmask. I use it a lot and I believe it can help with your problem. If I understand the problem correctly, it may be solved in the synopsis.

    -Paul

Re: Sorting IP Subnets together
by strat.minger (Initiate) on Apr 17, 2007 at 08:43 UTC
    Thanks for the help on this, I will give this a try and will let you know how I get on.

    Sorry for my lack of formatting in the original post:)
Re: Sorting IP Subnets together
by strat.minger (Initiate) on Apr 17, 2007 at 12:15 UTC
    Thanks again, it works! Once I figured out how to get my info into an array of arrays!!