Re^3: Grep logs by start date and end date in different directories
by Anonymous Monk on Jan 05, 2018 at 01:53 UTC
|
Sorry to bother again but I am kinda urgent on this so I am working this on my own but at the same time I hope i get more insights from professionals which can allow me to do it in a better way. My current script also searches for IP in a network range from all the log file. This is the code that does what I've mentioned:
use Net::Subnet;
if (@ARGV){
while (<>) {
my @ips = m/(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(
+?:25
+[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/g;
next unless @ips;
next unless grep { $matcher->($_) } @ips;
print $fh $_;
}
Do you know how I can implement this into your code? Thanks again | [reply] [d/l] |
|
|
# untested
use IO::Uncompress::Bunzip2 ();
my $fh = IO::Uncompress::Bunzip2->new($filename)
or die "bunzip2 $filename: $IO::Uncompress::Bunzip2::Bunzip2Error"
+;
# use $fh like a regular filehandle
As for the second question, it'd be best if you followed the advice in SSCCE and How do I post a question effectively? - especially if it's urgent, since the advice on those pages will help us provide you with help efficiently. Since the above questions aren't closely related to the original one, you may also want to post a new question in a new thread. Anyway, there have been a few recent threads with similar topics that you might be able to get some information from:
| [reply] [d/l] |
|
|
Hi haukex, thank you so much for your reply. Managed to get it to work. However what happens if I want to read multiple bz2 files instead of 1 bz2 file? My current code is:
my $filename = "debug.log.bz2";
my $fh = IO::Uncompress::Bunzip2->new($filename)
or die "bunzip2 $filename: $IO::Uncompress::Bunzip2::Bunzip2Error"
+;
while(<$fh>){
#do something
}
| [reply] [d/l] |
|
|
|
|
Hello Anonymous Monk,
Apologies for the late reply, but I just noticed your reply to my comment.
It is very open your questions, I am not sure what do you mean with My current script also searches for IP in a network range from all the log file. network range can vary greatly. Give a bit more specific information e.g. 127.0.0.1 - 127.0.0.255 what is the range, how the IP will be imported? I mean you will import IP e.g. 127.0.0.1 and you want to check what IP are matching what the network, subnet, range? On your log files is this exact IP exists? Or are you looking for any number that consists of 1-255.1-255.1-255.1-255?
We need sample of data in the files to see the format. For example you just mentioned that you are having bz2 files and fellow Monk haukex proposed a module and a few similar questions.
So help us with more specific information to help you.
Hope this helps, BR.
Seeking for Perl wisdom...on the process of learning...not there...yet!
| [reply] [d/l] [select] |
|
|
Hi again, sorry for the lack of information. What I meant is when user provide a network address with a CIDR range such as 192.168.1.0/27, all IP address under that subnet will be printed. As for the log files, there are IP addresses in every line of different logs. I could not provide the data because it is confidential. Thank you again
| [reply] |
|
|
Update, I managed to get the IP address subnet range searching to work. However, to implement it into your code, I am unsure why when i put ->name('*.bz2') it doesn't work.. This is my code currently:
use strict;
use warnings;
use IO::Uncompress::Bunzip2 ();
use Net::Subnet;
my $filename = "debug.log.bz2";
my $fh = IO::Uncompress::Bunzip2->new($filename)
or die "bunzip2 $filename: $IO::Uncompress::Bunzip2::Bunzip2Error"
+;
my $matcher = subnet_matcher qw(72.46.130.0/24);
while (<$fh>){
my @ips = m/(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25
+[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/g;
next unless @ips;
next unless grep { $matcher->($_) } @ips;
print;
}
And this is your code that I am unsure of how to search for all .bz2 files:
sub get_files {
my (@dirs) = @_;
my $level = shift // 3; # level to dig into
my @files = File::Find::Rule->file()
->name('*.bz2') #this line doesn't work
->maxdepth($level)
->in(@dirs);
return @files;
}
| [reply] [d/l] [select] |
|
|
Hi, please ignore all my replies above as I have already solved it myself (which I am amazed cause I suck at programming)... However, I have one last question, for this part of the code:
sub get_files {
my (@dirs) = @_;
my $level = shift // 3; # level to dig into
my @files = File::Find::Rule->file()
->name( '*.bz2' ) #can insert regex too
->maxdepth($level)
->in(@dirs);
return @files;
}
This particular line:
my (@dirs) = @_;
When I change it to my own log file path, it will replace the <date>/<filename.bz2> which will returns nothing in result. How can I specify my own log path? | [reply] [d/l] [select] |
|
|
| [reply] |
|
|