http://qs1969.pair.com?node_id=723734

JPaul has asked for the wisdom of the Perl Monks concerning the following question:

Greetings all;

I have a need to scratch and haven't been able to work out a good way of doing it all together in an elegant, let alone functional, way.
I need to capture and count packets coming from multiple sources, using multiple filters, in real time. Right now I just have three separate perl scripts doing the task, but I'd like to fold it into a single tool. I currently use an open() on the tcpdump tool (under Linux).

Simple psuedo-code to give you an idea what I'm up to: 

$SIG{ALRM} = \&every_sec();  alarm(1);

open(PROCESS, "tcpdump -ln 'filter'"|);
while(<PROCESS>) {
    next if (!&ParseIPandCheckSomething($_));
    $Counter++;
    }

sub every_sec {
    print "This past second: $Counter\n";
    $Counter = 0;
    alarm(1);
    }

[download]
Here are the approaches I've looked at:
Incorporating Net::Pcap and doing it the "proper" way without reading the external 'tcpdump' output... except Net::Pcap can't handle multiple instances (the filters and like are stored in a single shared variable, according to its documentation). I also require the use of a SIGALRM to keep track of per-second counts, and Net::Pcap::loop must disable this, because the alarm callback certainly isn't being called each second like it's supposed to.

Opening several 'tcpdump' filehandles and looping over reading each one. When I do this the 'real time' aspect of it goes to heck. I was trying to do a bunch of non-blocking read()s and pull in a block of bytes, jump to the next file handle, and so on, and then process completed inputted packets... but it obviously isn't either fast enough, or designed properly, because "real time" it is not.

foreach $filter (qw(filter1 filter2 filter3)) {
    my $fh = FileHandle->new();
    $fh->open("tcpdump -ln $filter|");
    $fh->autoflush(1);
    push(@FHStack, $fh);
    }

while(1) {
    # Read anything (up to 4096 bytes) from tcpdump
    foreach my $fh (@FHStack) {
         my $bytes = read($FH, my $buffer, 4096);
         $PackStack{$FH} .= $buffer;
         }

    # Process what we've gotten, looking for a complete packet
    foreach my $fh (@FHStack) {
         if ($PackStack{$FH} =~ /\n/) {
              # LF means completed packet received from tcpdump
              $PackStack{$FH} = &ProcessPacketAndRemove($PackStack{$FH
+});
              }
         }
     }

[download]
Boy, folks, I know there's a better way to do this, I'd sure love to hear your input.

Thanks!

-- Alexander Widdlemouse undid his bellybutton and his bum dropped off --

Replies are listed 'Best First'.
Re: Real time reading of multiple pcap 'instances' at once
by Fletch (Bishop) on Nov 15, 2008 at 00:24 UTC

    It's a bit long in the tooth now, but you probably could use POE::Component::Pcap (which sits on top of Net::Pcap) and let POE do the nasty socket-y-select-y bits.

    Update: Hrmm, wait I glossed over where you need multiple pcap filters. Perhaps you could use all the filters as conditions (joined with 'or') and then figure out which hit for a given packet in your code?

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

[reply]
      I was about to dismiss this as a silly idea, but the voice in the back of my head went "You know, the obvious answer might not be quite as daft as it first seems".

      Although, due to what I'm trapping, I can't really pull the packets into perl and use it to work it out - they do happen to be, currently, all on different ports - which, of course, tcpdump clearly shows. I don't even need to mess with Net::Pcap and it's screwing with SIGALRM - I can simply use my existing implementation with a few extra nuts and bolts. I think.

      Thank you for the probably obvious, but nevertheless gratefully received, suggestion.

      -- Alexander Widdlemouse undid his bellybutton and his bum dropped off --
[reply]
Re: Real time reading of multiple pcap 'instances' at once
by MidLifeXis (Monsignor) on Nov 14, 2008 at 22:18 UTC

    Perhaps select might be what you are looking for?

    --MidLifeXis

[reply]
      For some odd reason I was simply -unable- to get select() to work properly on filehandles that encapsulated a process. select() was actually my first port of call, and after spending several hours without luck, I switched to the method outlined above.
      Unfortunately it's been a few weeks since I tried the select() method, so I do not recall the specifics - I've used it a thousand times with TCP sockets, so it's not like I've never used it before... I'm sorry, I just can't remember why it didn't work properly. I'll see if I can dig up the source code to remind myself!

      Thank you, however.

      -- Alexander Widdlemouse undid his bellybutton and his bum dropped off --
[reply]
Re: Real time reading of multiple pcap 'instances' at once
by Corion (Patriarch) on Nov 15, 2008 at 12:24 UTC

    At least under Linux or other unixish OSes, the select approach of reading from multiple tcpdump instances should work. Otherwise, I'd aim for the mod//Net::Pcap approach, using pcap_loop() to read some packets. Also, consider combining the "queries" you hand to the different Pcap instances into one large query. For example, if you want to capture (HTTP) traffic between the local machine and two different hosts, use the following specification:

        (dest www1.example.com && (tcp port 80))
  ||(src www1.example.com  && (tcp port 80))
  ||(dest www2.example.com && (tcp port 80))
  ||(src www2.example.com  && (tcp port 80))

    [download]

    So if you have traffic going over different (known) ports to different (known) machines/IP addresses, the simplest approach might be to capture them all in one stream and sort them out in your Perl code again.

[reply]
[d/l]
[select]
Re: Real time reading of multiple pcap 'instances' at once
by zentara (Archbishop) on Nov 15, 2008 at 19:57 UTC
[reply]

Back to Seekers of Perl Wisdom