Netflow data, hmm? I've dealt with that, and the traffic can get pretty heavy. Your best bet is to make not exactly a multithread server, but a server thread that accepts each packet, and as quickly as possible puts it somewhere to be processed. (This could be through shared memory, pipes, databases or files - But SHM and pipes just removes the problem one layer deeper).

In a multithreaded server, that means one receiving thread, and possibly multiple processing threads. Or you could have a singlethreaded server which simply stores the info somewhere to be processed.

Where you store it depends on how many packets we're talking about, how much space you have to store them, and how much pre-processing you intend to do.

If you're getting a significant ammount of netflow (we get a lot even at 300:1 sampling) I would advise against storing it in a database directly - they are just not fast enough. You could use a system of rotating DB_File databases, or just dump to a file, and process it later.

HTH.

thanks for your reply. I will try to write a server thread that accepts each packet, and as quickly as possible puts it somewhere to be processed. And I will dump the data to a file. And do you use other language to process netflow data like c or c++ ? I am in doubt whether perl can process this heavy data perfectly. thanks very much!

We're using a C program to capture the flows and dump them to a file. Then we use a Perl program to filter the flows and look for interesting stuff.

If you're going to be parsing the flows in Perl, I strongly suggest you take a good look at Dave Plonka's CFlow perl module.

you want http://www.splintered.net/sw/flow-tools. much, much, much faster than Perl for dealing with netflow.

I would turn to proven solutions instead of writing a multithreaded solution in Perl for network sniffing:

• Net::PCap and Net::PCapUtils are interfaces to libpcap, a capturing library. libpcap also does buffering, so you could be able to get away without losing data when writing.
• Use ethereal to capture the data and process the capture files afterwards through Perl. This has the advantage of using a tried and true network sniffer, relieving you of all the hassles.