iptables log auditing

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi team, I am wanting to monitor network and user activity. For example how much bandwidth a user has used, what websites they have visited etc. This is a request from my boss. I am developing a linux server running iptables. I apologise if this is not directly related to perl, but the wisdom here is second to none. I have looked at tools such as iptraf and thought about writing a perl script to manipulate the log file into a report. The monitoring is to be done on a per machine bases (would be interesting to note techniques for per user basis). What techniques are widely adopted by others? Can scripts be written for iptables to do the same purpose. Here in New Zealand we have tight bandwidth caps. Your guidance and advise would be most appreciated. Thank you.

Comment on iptables log auditing

Replies are listed 'Best First'.
Re: iptables log auditing by sh1tn (Priest) on Jun 21, 2005 at 12:32 UTC
Iptables modules	[reply]
Re: iptables log auditing by monarch (Priest) on Jun 21, 2005 at 13:17 UTC
The iptables has a clear (or zero) counters command. Have a good read of the iptables manual which will give you command line options (including one which reports in numerical format that you will find easier for parsing). Possibly one approach would be to run a daily cron job script. That script would query iptables for the current counters, and grep for the counters you want. It may then immediately call iptables again with the zero counters command. There are, of course, hundreds of other approaches but that would be one way of approaching it. (I'm assuming you know how to configure iptables and set up firewall rules/chains.. if not then you really are out of your depth and your time is best spent googling more information on iptables)	[reply]
Re: iptables log auditing by tirwhan (Abbot) on Jun 21, 2005 at 13:20 UTC
If you use the iptables ULOG target (available in kernels >2.4.18) and ulogd daemon you can log raw packets to either a log file or directly into a mysql database. This gives you much more flexibility in terms of the data you want to log and also on how you want to handle the packets afterwards (just use one of the libpcap modules). See http://www.sunbeam.franken.de/projects/ulogd for more details on ulogd. However, from your description of the goals you are trying to achieve it sounds like you're looking for netflows. Take a look at fprobe and flow-tools (the latter comes with a perl module for further processing).	[reply]
Re: iptables log auditing by Anonymous Monk on Jun 22, 2005 at 13:52 UTC
Thanks alot for the advise. I do have experience with iptables, so i will look into the techniques mentioned.	[reply]
Re: iptables log auditing by Anonymous Monk on Jan 16, 2007 at 10:59 UTC
Thought i would add an update to this thread. I ended up using ipfm and a custom perl script to manipulate. This gives me total up/down bandwidth on a per ip basis. I am still working on a better solution though. Although this is a relatively small network (25 users) reserving ip addresses for auditing does not seem right. I would much rather have the ability to track usage on a per user basis. I am unsure whether this is viable using Linux. Any thoughts to add would be great	[reply]