in reply to newbie question about mulit-thread udp server

Well, for first approximation, the UDP server is not much different from the TCP server = it's only simpler, since all of the information from each connection is in the first packet.

Netflow data, hmm? I've dealt with that, and the traffic can get pretty heavy. Your best bet is to make not exactly a multithread server, but a server thread that accepts each packet, and as quickly as possible puts it somewhere to be processed. (This could be through shared memory, pipes, databases or files - But SHM and pipes just removes the problem one layer deeper).

In a multithreaded server, that means one receiving thread, and possibly multiple processing threads. Or you could have a singlethreaded server which simply stores the info somewhere to be processed.

Where you store it depends on how many packets we're talking about, how much space you have to store them, and how much pre-processing you intend to do.

If you're getting a significant ammount of netflow (we get a lot even at 300:1 sampling) I would advise against storing it in a database directly - they are just not fast enough. You could use a system of rotating DB_File databases, or just dump to a file, and process it later.

HTH.

  • Comment on Re: newbie question about mulit-thread udp server and documention of io::socket

Replies are listed 'Best First'.
Re: Re: newbie question about mulit-thread udp server and documention of io::socket
by iwanthome (Beadle) on Mar 24, 2004 at 08:19 UTC
    thanks for your reply. I will try to write a server thread that accepts each packet, and as quickly as possible puts it somewhere to be processed. And I will dump the data to a file. And do you use other language to process netflow data like c or c++ ? I am in doubt whether perl can process this heavy data perfectly. thanks very much!
      We're using a C program to capture the flows and dump them to a file. Then we use a Perl program to filter the flows and look for interesting stuff.

      If you're going to be parsing the flows in Perl, I strongly suggest you take a good look at Dave Plonka's CFlow perl module.