Re: Extracting data from a firewall log

Here is a snip from the firewall log, of what I am attempting to extract from the log:
03/13/03 16:44:56 kernel Temporarily blocking host 212.241.116.21 03/13/03 16:44:57 firewalld[103] deny in eth0 48 tcp 20 117 212.241.11 +6.21 209.126.xxx.xxx 4449 80 syn (LO-Proxied-HTTP)
[download]

This must mean that you are able somehow to read the logs: let's assume that you did so by opening the wgl-log file in some sort of editor (and not in a proprietary viewer).

There are strong chances then that the log is in (a variant of) ASCII and Perl will be able to read the log by opening a read-filehandle and inputting the log line-by-line through us of the <INPUT-FILEHANDLE> function.

If all log-lines start with a date-and-timestamp, you can extract all data following these and put it through some regular expressions to weed out the useless entries and keep the valuable ones, which you can then either output to another file, save in a database, calculate some statistics from or --in general-- mangle beyond all recognition to your (and Perl's) heart's content.

To parse the logfile, you might have a look at regexp-log, HTTPD-Log-Filter or Log-Detect. Even if you can't use these modules directly, they will certainly give you some good ideas on how to tackle your task!

CountZero

"If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Comment on Re: Extracting data from a firewall log Select or Download Code

Replies are listed 'Best First'.
Re: Re: Extracting data from a firewall log by tarballed (Initiate) on Apr 09, 2003 at 19:40 UTC
Yes, that is the correct. I have sent the logs over to my Linux server via syslogd. The files are now viewable with any text editor. Also, I am planning on searching through the file, then outputting the results to another file and save in a database.(My list just continues to grow.) I have to say, this task seems very daunting to me. If I could trouble someone, could you get me started a little? Maybe give me a few hints or examples. I think once I have a few things under my belt, I will feel more more confident and where to proceed from that point. Thanks. tarballed	[reply]
Re: Re: Re: Extracting data from a firewall log by CountZero (Bishop) on Apr 09, 2003 at 20:14 UTC
For starters, you can have a look at Basic Input and Output and File Input and Output or Re: How do I read the contents of a file?. For regular expressions String matching and Regular Expressions and the items linked to it, are surely a must. Of course, you have already read Common Beginner Mistakes and will heed its warnings. CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law	[reply]