I'm using Perl to write a script that will parse the log(s) of some intrusion detection software that we are runnnig. Since the software runs the netrok cards in promiscuous, there are a lot of entries in the logs that don't pertain to our box. Luckily, the software explicitly logs source and destination IP addresses. Unluckily, it only does it for connections. Raw ASCII data captures do not include full information. Here is a snippet from the log (various things have been removed for clarity): 
*snip* incoming connection from=(111.111.111.111:1234) to=(22.22.22.22
+:80)
*snip* ASCII data in TCP packet from=(111.111.111.111:1234), localport
+=(80), *data here*

[download]

What I planned to do was parse the IP:port pairs for connections, and place the data for connections not to our box into a hash. Then I would use the information in the hash as an "ignore" list. The only problem I can see is that I have no way of knowing when a certain connection has completed and I can safely remove the entry from the hash.
To be more explicit - if the log were to look like this:
*snip* incoming connection from=(111.111.111.111:1234) to=(22.22.22.22
+:80)
*snip* ASCII data in TCP packet from=(111.111.111.111:1234), localport
+=(80), *data here*
*snip* incoming connection from=(111.111.111.111:1234) to=(my.box.ip.a
+ddr:80)
*snip* ASCII data in TCP packet from=(111.111.111.111:1234), localport
+=(80), *data here*

[download]

I would ignore the connection from 111.111.111.111:1234 to 22.22.22.22:80 since that connection is not destined for my box. But, since the ASCII capture only catches the destination port, the second captured ASCII packet would be discarded since the source IP is listed as being ignored.

My question (finally!) is this: how do I construct the hash and populate and remove entries so that I can accurately reflect connections that I can ignore? Is this even something that can be done easily? Does my question even make sense?


Guildenstern
Negaterd character class uber alles!

In reply to Algorithmic difficulties by Guildenstern

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.