Dru has asked for the wisdom of the Perl Monks concerning the following question:

Monks,

I'm in need of some assistance. I have the following data:
------------------------------------------------- Host: 192.168.1.202 Open ports: Service: http (80/tcp) Severity: Low The IIS server appears to have the .IDA ISAPI filter mapped. At least one remote vulnerability has been discovered for the .IDA (indexing service) filter. This is detailed in Microsoft Advisory MS01-033, and gives remote SYSTEM level access to the web server. ------------------------------------------------- Host: 192.168.1.21 Open ports: Service: https (443/tcp) Severity: High The remote host seems to be using a version of OpenSSL which is older than 0.9.6e or 0.9.7-beta3 This version is vulnerable to a buffer overflow which, may allow an attacker to obtain a shell on this host. Service: https (443/tcp) Severity: Low The remote host is using a version of OpenSSL which is older than 0.9.6j or 0.9.7b This version is vulnerable to a timing based attack which may allow an attacker to guess the content of fixed data blocks and may eventually be able to guess the value of the private RSA key of the server. ------------------------------------------------- Host: 192.168.1.22 Open ports: Service: http (80/tcp) Severity: High It might be possible to make the remote IIS server execute arbitrary code by sending it a too long url ending in .htr.
That I would like to format as such:
------------------------------------------------- Vulnerability: Service: http (80/tcp) Severity: High It might be possible to make the remote IIS server execute arbitrary code by sending it a too long url ending in .htr. Hosts: 192.168.1.22 192.168.1.202 ------------------------------------------------- Vulnerability: Service: https (443/tcp) Severity: High The remote host seems to be using a version of OpenSSL which is older than 0.9.6e or 0.9.7-beta3 This version is vulnerable to a buffer overflow which, may allow an attacker to obtain a shell on this host. Hosts: 192.168.1.21
Below is my code attempt. I'm thinking I need a hash with the key being the vulnerability and the value's being the ip's. I got the code for the hash here: Unique keys for multiple values with a Hash, but it's not working(nothing happens). The other problem is my regex. It works fine when a host only has one vulnerability to it, but not against multiple vulnerabilities (like 192.168.1.21 above) , I'm not sure the best way to write it to catch these.

My apologizes for the long post.
Dru
use strict; use warnings; my $file = 'input.txt'; my (%hash, @ips, @alerts); open (FILE, "$file") or die "Can't open $file\n"; while (<FILE>){ $/ = '-------------------------------------------------'; $hash{$2}{1} = 0 if (/(Host: \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).* (Service:.*Severity:.*)/ms); } foreach my $key (keys %hash) { $hash{$key} = [keys %{$hash{$key}}]; }

Replies are listed 'Best First'.
Re: Building a Hash with Multiple Values
by graff (Chancellor) on Aug 05, 2004 at 05:31 UTC
    It looks like you have a set of many-to-many relations: some hosts have multiple services with vulnerabilities, and some of the vulnerabilities show up on multiple hosts. You are given an input list that organizes the relations by host, and you want to create the "inverse" list, organized by vulnerability.

    If that's correct, then I think you don't quite have the right data structure yet, and your loop is not giving proper treatment to hosts with multiple "Service:..." entries -- e.g. suppose host A has service problems X and Y, host B has just X, and host C has just Y. The way the OP code is written, the output list will have three "vulnerability" units, with a single host in each unit. If the idea is to group all hosts that have a given problem, you missed that goal. Something like the following might work better:

    use strict; use warnings; $/ = "-------------\n"; # don't bother counting dashes my %services; # keys will be service messages, values will be arrays o +f host IPs while (<>) # put file name on command line (or pipe data to stdin) { my @service_keys = (); my $current_service = ''; my @lines = split /\n/; # step through line-by-line... my $host = ''; while ( @lines ) { $_ = pop @lines; # ... from the bottom up next if /-------/; $current_service = "$_\n$current_service"; if ( /^Service: / ) { # top of a sub-record push @service_keys, $current_service; $current_service = ''; } elsif ( /^Host: +([\d.]+)/ ) { # top of record $host = $1; push @{$services{$_}}, $host for ( @service_keys ); last; } } warn "Record $. did not contain a host IP\n" # (update: die is to +o harsh here) if ( $host eq '' ); # just in case... } # %services is now HoA; let's suppose we want # to order the output by "severity" (high ... low) # but within any severity level, order doesn't matter: for my $level ( qw/High Moderate Low/ ) # any others? { for my $service ( grep /Severity: $level/, keys %services ) { print "-----------------\n\nVulnerability:\n\n$service\nHosts: +\n"; print join "\n", @{$services{$service}}, "\n"; } }
    This seems to work, but I only tried it on the sample of input you provided, which didn't have any cases of two hosts with the same problem.

    One thing you might need to be careful about is variable white-space in the input service reports -- e.g. if a given message is reported on various hosts with differences in blank lines or trailing spaces, it will show up as multiple keys in the main hash. But when you start by reading a whole record at a time like you're doing, it's easy to start each iteration by normalizing white-space, if necessary.

    (update: when I fudged an extra "Host:" entry to try for two hosts with the same problem, I discovered that the last record in the input file needs to end with "------\n", or else the "split /\n/" needs to be  split /\n/,$_,-1, to preserve trailing newlines, if any; and/or you need to muck with normalizing whitespace anyway.)

      Wow! That's exactly what I was trying to do. Thanks to you and the other's who replied. I need to sit down and go through this code line by line and I'm sure I will learn a great deal. I have not learned HoA's yet, so this will be a good time to start.

      I know you spent some time on this reply and I am one very grateful monk. I'm going to donate $25 to the monstary in your name.
Re: Building a Hash with Multiple Values
by steves (Curate) on Aug 05, 2004 at 03:48 UTC

    The line break in your regexp is throwing off your match. If you make that a single line it ends up getting matches. What you end up with is a hash where the keys are "Service ..." all the way to the end of the text block and the values are "Host: <ip>" strings. So your keys are multi-line blocks of text and your values are strings that have IP's. That doesn't seem to be what you'd really want here but maybe it's a start.

    You can also put that $/ setting once outside the loop. I find it good practice to local those but it doesn't matter much for a standalone piece of code.

    I tested this version:

    use strict; use warnings; my $file = 'input.txt'; my (%hash, @ips, @alerts); open (FILE, "$file") or die "Can't open $file\n"; local $/ = '-------------------------------------------------'; while (<FILE>){ $hash{$2}{$1} = 0 if (/(Host: \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*(Service:.*Sever +ity:.*)/ms); } foreach my $key (keys %hash) { $hash{$key} = [keys %{$hash{$key}}]; } my ($key, $values); while (($key, $values) = each %hash) { print "$key --> ", join('|', @$values), "\n"; }

Re: Building a Hash with Multiple Values
by davido (Cardinal) on Aug 05, 2004 at 04:47 UTC

    If you wish to embed whitespace (newlines, spaces, tabs), and '#' initiated comments in a regular expression, you should end the regexp with the /x modifier. See perlre for details.

    The /x modifier is very useful because it makes it easy to break regular expressions into visually recognizable chunks, possibly spanning multiple lines.


    Dave