|
Re^4: Grep logs by start date and end date in different directories
by haukex (Archbishop) on Jan 05, 2018 at 09:14 UTC
|
Regarding decompressing, there is the core module IO::Uncompress::Bunzip2 that you should be able to use fairly transparently:
# untested
use IO::Uncompress::Bunzip2 ();
my $fh = IO::Uncompress::Bunzip2->new($filename)
or die "bunzip2 $filename: $IO::Uncompress::Bunzip2::Bunzip2Error"
+;
# use $fh like a regular filehandle
As for the second question, it'd be best if you followed the advice in SSCCE and How do I post a question effectively? - especially if it's urgent, since the advice on those pages will help us provide you with help efficiently. Since the above questions aren't closely related to the original one, you may also want to post a new question in a new thread. Anyway, there have been a few recent threads with similar topics that you might be able to get some information from:
| [reply]
[d/l] |
|
|
Hi haukex, thank you so much for your reply. Managed to get it to work. However what happens if I want to read multiple bz2 files instead of 1 bz2 file? My current code is:
my $filename = "debug.log.bz2";
my $fh = IO::Uncompress::Bunzip2->new($filename)
or die "bunzip2 $filename: $IO::Uncompress::Bunzip2::Bunzip2Error"
+;
while(<$fh>){
#do something
}
| [reply]
[d/l] |
|
|
However what happens if I want to read multiple bz2 files instead of 1 bz2 file?
You should just be able to put the code you showed in a loop, for example my @files = ("debug.log.bz2", "debug2.log.bz2", ...); for my $filename (@files) { my $fh = ... }.
Update: If you're the same anonymous poster, then you seem to have already figured that out here. You might want to consider registering an account so that your threads can be more easily connected, and so that you can edit your posts instead of making multiple anonymous "update" posts. Even if you already have an account you can still register a second one for use at work as long as you observe the Site Rules Governing User Accounts.
| [reply]
[d/l] |
Re^4: Grep logs by start date and end date in different directories
by thanos1983 (Parson) on Jan 05, 2018 at 12:37 UTC
|
Hello Anonymous Monk,
Apologies for the late reply, but I just noticed your reply to my comment.
It is very open your questions, I am not sure what do you mean with My current script also searches for IP in a network range from all the log file. network range can vary greatly. Give a bit more specific information e.g. 127.0.0.1 - 127.0.0.255 what is the range, how the IP will be imported? I mean you will import IP e.g. 127.0.0.1 and you want to check what IP are matching what the network, subnet, range? On your log files is this exact IP exists? Or are you looking for any number that consists of 1-255.1-255.1-255.1-255?
We need sample of data in the files to see the format. For example you just mentioned that you are having bz2 files and fellow Monk haukex proposed a module and a few similar questions.
So help us with more specific information to help you.
Hope this helps, BR.
Seeking for Perl wisdom...on the process of learning...not there...yet!
| [reply]
[d/l]
[select] |
|
|
Hi again, sorry for the lack of information. What I meant is when user provide a network address with a CIDR range such as 192.168.1.0/27, all IP address under that subnet will be printed. As for the log files, there are IP addresses in every line of different logs. I could not provide the data because it is confidential. Thank you again
| [reply] |
|
|
Update, I managed to get the IP address subnet range searching to work. However, to implement it into your code, I am unsure why when i put ->name('*.bz2') it doesn't work.. This is my code currently:
use strict;
use warnings;
use IO::Uncompress::Bunzip2 ();
use Net::Subnet;
my $filename = "debug.log.bz2";
my $fh = IO::Uncompress::Bunzip2->new($filename)
or die "bunzip2 $filename: $IO::Uncompress::Bunzip2::Bunzip2Error"
+;
my $matcher = subnet_matcher qw(72.46.130.0/24);
while (<$fh>){
my @ips = m/(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25
+[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/g;
next unless @ips;
next unless grep { $matcher->($_) } @ips;
print;
}
And this is your code that I am unsure of how to search for all .bz2 files:
sub get_files {
my (@dirs) = @_;
my $level = shift // 3; # level to dig into
my @files = File::Find::Rule->file()
->name('*.bz2') #this line doesn't work
->maxdepth($level)
->in(@dirs);
return @files;
}
| [reply]
[d/l]
[select] |
|
|
Hi, please ignore all my replies above as I have already solved it myself (which I am amazed cause I suck at programming)... However, I have one last question, for this part of the code:
sub get_files {
my (@dirs) = @_;
my $level = shift // 3; # level to dig into
my @files = File::Find::Rule->file()
->name( '*.bz2' ) #can insert regex too
->maxdepth($level)
->in(@dirs);
return @files;
}
This particular line:
my (@dirs) = @_;
When I change it to my own log file path, it will replace the <date>/<filename.bz2> which will returns nothing in result. How can I specify my own log path? | [reply]
[d/l]
[select] |
|
|
| [reply] |
|
|
| [reply]
[d/l]
[select] |