Which Data Structure Should I Be Using?

dru145 has asked for the wisdom of the Perl Monks concerning the following question:

Monks,

I consider myself fairly decent at Perl, but when it comes to what I consider complex data structures (hash of hashes, hashes of arrays), I'm lost. I've been reading Chapter 9: Data Structures of the Camel for the past few days now trying to avoid posting here, but I finally broke down. I have the following data from a portscanning program:

* + 192.168.99.199   salamander.acme.com
        |___   135  DCE endpoint resolution
        |___   139  NETBIOS Session Service
        |___  1031  BBN IAD
        |___  5800  Virtual Network Computing server
        |___  5900  Virtual Network Computing server
                |___ RFB 003.003.
* + 192.168.99.131   AMSPWD099WKG
        |___   135  DCE endpoint resolution
        |___   139  NETBIOS Session Service
        |___   445  Microsoft-DS
        |___  1027  ICQ?
        |___  1099  BBN IAD
* + 192.168.99.133   VLW4S
        |___   139  NETBIOS Session Service
* + 192.168.99.136   WKBX0010B-A
        |___    80  World Wide Web HTTP
                |___ HTTP/1.1 200 OK..Server: Microsoft-IIS/5.0..Date:
+ Thu, 14 Mar 2002 14:37:54 GMT..Connection: Keep-Alive..Content-Lengt
+h: 
1270..
        |___   135  DCE endpoint resolution
        |___   139  NETBIOS Session Service
        |___   443  https  MCom
        |___   445  Microsoft-DS
        |___  1058  nim
        |___  1059  nimreg
        |___  1433  Microsoft-SQL-Server
        |___  5003  Claris FileMaker Pro
[download]

I wrote a CGI interface on a secure internal webserver that allows my team to upload a file and add the data to a MySQL DB via DBI. I'm not having any problems with .csv files, but this one is throwing me for a loop. I contemplated telling the team only to use programs that have the ability to produce .csv files, but that limits them to only a few useless programs and I would like to figure this out anyhow. I'm thinking the easiest way to add this data to the MySQL DB would be to build a hash that uses the ip as the key and the rest of the data will be the values, but then I have the scenario, where the banner for certain ports (ie. 80) have to stay with that port, so I guess I need another hash that uses the port number as the key and the banner as the value? Well, I know this is pretty far off, but here is my attempt at sorting out this mess. I would appreciate any assistance that my fellow monks could provide:

#!/usr/bin/perl -w
 
use strict;
 
my $infile = './portscan2.txt';
my ($ip, @fields, @cname, @ports, @banners);
my @field_names = qw(cname port banner);
my %data;
 
# Change the input record separator
local $/ = "* +";
 
 
open INFILE, "$infile" or die "Can't open $infile: $!\n";
while (<INFILE>){
 next unless /\b(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b/;
 $ip = $1;
 push (@cname, $1) if /\b$ip\b\s{2,}(\w[^\s]*|\[Unknown\])\s{2}/;
 push (@ports, $1) if /((\d{1,5}?)\s{2}(\w[^\n]*))\s{2}/;
 push (@banners, $1) if /\Q|___\E\s+([^"]*)\.{1,3}\s+\n/;
 
 my $fields = [@cname, @ports, @banners];
 
 @{$data{$ip}}{@field_names} = [$fields];
}
 
foreach my $ip (keys %data) {
  print "IP = $ip, Computer Name = $data{$ip}{cname}->[0][0]\n";
}
 
close INFILE;
[download]

This prints:

IP = 192.168.99.136, Computer Name = salamander.acme.com
IP = 192.168.99.199, Computer Name = salamander.acme.com
IP = 192.168.99.131, Computer Name = salamander.acme.com
IP = 192.168.99.133, Computer Name = salamander.acme.com
[download]

Thanks,
Dru
Another satisfied monk.

Comment on Which Data Structure Should I Be Using? Select or Download Code

Replies are listed 'Best First'.
Re: Which Data Structure Should I Be Using? by dragonchild (Archbishop) on Apr 02, 2002 at 16:32 UTC
`@{$data{$ip}}{@field_names} = [$fields];` [download] That line is an error. You're saying that a slice of a hash reference is set equal to an array reference (whose only element is an array reference, I might add). Better would be: `@{$data{$ip}}{@field_names} = @$fields;` [download] Now, you still have a problem with `my $fields = [@cname, @ports, @banners];` [download] You're saying that you want to create an array reference, pointing to an array of elements from the three arrays in order. I suspect you really want: `my @fields = (\@cname, \@ports, \@banners);` [download] Thus, the complete code would be: `my @fields = (\@cname, \@ports, \@banners); @{$data{$ip}}{@field_names} = @fields;` [download] That should do what you want. :-) ------ We are the carpenters and bricklayers of the Information Age. Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.	[reply] [d/l] [select]
Re: Re: Which Data Structure Should I Be Using? by dru145 (Friar) on Apr 02, 2002 at 20:03 UTC
Thanks Dragonchild, I'm happy to know that I was closer then I thought. I'm trying to print out each value with this: `foreach my $ip (keys %data) { print "IP = $ip\n"; for my $cname ( @{$data{$ip}{cname}} ) { print "CName = $cname\n"; } }` [download] and here is what I get: `IP = 192.168.30.136 CNAME = salamander.acme.com CNAME = AMSPWD030WKG CNAME = VLW4S CNAME = WKBX0010B-A IP = 192.168.30.130 CNAME = salamander.acme.com CNAME = AMSPWD030WKG CNAME = VLW4S CNAME = WKBX0010B-A IP = 192.168.30.131 CNAME = salamander.acme.com CNAME = AMSPWD030WKG CNAME = VLW4S CNAME = WKBX0010B-A IP = 192.168.30.133 CNAME = salamander.acme.com CNAME = AMSPWD030WKG CNAME = VLW4S CNAME = WKBX0010B-A` [download] How do I keep the cname value with the ip? Thanks, Dru Another satisfied monk.	[reply] [d/l] [select]
Re3: Which Data Structure Should I Be Using? by dragonchild (Archbishop) on Apr 02, 2002 at 20:36 UTC
You are "Suffering from Scoping". (As well as two rather nasty side-effects of references, but those'll get fixed.) Let's diagram what you're doing with @cname. You've declared it at the top of your program. This implies that you are attempting to gather together a list of computer names from across your entire file. This doesn't sound like what you want to do. Instead, you want to gather a list of the computer names for a given IP address. Again, you're almost there. `push @{$data{$ip}{cname}}, $1 if /\b$ip\b\s{2,}(\w[^\s]\|\[Unknown\])\s{2}/; push @{$data{$ip}{ports}}, $1 if /((\d{1,5}?)\s{2}(\w[^\n]))\s{2}/; push @{$data{$ip}{banners}}, $1 if /\Q\|___\E\s+([^"])\.{1,3}\s+\n/;` [download] This way, you're doing the pushing onto the array you actually want. (Obviously, you remove the declaration of @cname and his friends, cause it's not needed anymore. You also remove the hash-slicing cause it's also not needed anymore.) For your personal edification, what was happening was that you were creating one array - @cname. Every instance of your data was referencing this specific array. Every time you modified it, each instance would know about the modifications. Every time you de-referenced it, you were de-referencing the same array. ------ We are the carpenters and bricklayers of the Information Age.* Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.	[reply] [d/l]