Hello Anonymous Monk,

Welcome to the Monastery. Fellow Monks have provided you with answers but I found your question interesting so I spend some time to wrote a small script that if I understand correctly from your description should do exactly what you want.

Sample of code:

#!/usr/bin/perl
use strict;
use warnings;
use Date::Manip;
use Data::Dumper;
use File::Find::Rule;

sub get_files {
    my (@dirs) = @_;

    my $level = shift // 2; # level to dig into

    my @files = File::Find::Rule->file()
    ->name('access.log', 'sys.log')
    ->maxdepth($level)
    ->in(@dirs);

    return @files;
}

sub searchForIP {
    my ($files, $ip) = @_;
    local @ARGV = @$files;

    while (<>) {
    print "$ARGV:$.:$_" if /$ip/;
    } continue {
    close ARGV if eof;
    }
    return;
}

my $numberOfDays = '2 days';
my $dateStart = ParseDate("today");
my $dateEnd = DateCalc($dateStart, $numberOfDays);

# To find the every day date1 to date2
my @dates =ParseRecur("0:0:0:1:0:0:0","",$dateStart, $dateEnd);

my @datesFormatted = map { UnixDate($_, '%Y-%m-%d') } @dates;
# print Dumper \@datesFormatted;

my @files = get_files(@datesFormatted);

# print Dumper \@files;

my $ip = "127.0.0.1";
searchForIP(\@files, $ip);

__END__

$ perl test.pl
2018-01-03/access.log:1:127.0.0.1 This is insident 1 in 2018-01-03
2018-01-03/access.log:4:127.0.0.1 This is second insident 4 in 2018-01
+-03
2018-01-05/sys.log:1:127.0.0.1 This is insident 1 in 2018-01-05
2018-01-05/sys.log:4:127.0.0.1 This is second insident 4 in 2018-01-05

[download]

I used the modules Date::Manip for the date calculation, File::Find::Rule to traverse the directories and get the files (you could have used the core module File::Find) and finally the debugging module Data::Dumper.

Data that I used to get the output that I am showing:

$ ls -la
total 40
drwxr-xr-x 8 tinyos tinyos 4096 Jan  3 11:37 .
drwxr-xr-x 5 tinyos tinyos 4096 Jan  2 20:38 ..
drwxr-xr-x 2 tinyos tinyos 4096 Jan  3 10:01 2018-01-01
drwxr-xr-x 2 tinyos tinyos 4096 Jan  3 10:02 2018-01-02
drwxr-xr-x 2 tinyos tinyos 4096 Jan  3 11:33 2018-01-03
drwxr-xr-x 2 tinyos tinyos 4096 Jan  3 10:02 2018-01-04
drwxr-xr-x 2 tinyos tinyos 4096 Jan  3 11:34 2018-01-05
drwxr-xr-x 2 tinyos tinyos 4096 Jan  3 11:27 2018-01-06
-rw-r--r-- 1 tinyos tinyos 1230 Jan  3 11:37 test.pl
-rw-r--r-- 1 tinyos tinyos  414 Jan  3 10:26 test.pl~

[download]

Each directory contains two files same as your description.

$ ls -la 2018-01-01/
total 8
drwxr-xr-x 2 tinyos tinyos 4096 Jan  3 10:01 .
drwxr-xr-x 8 tinyos tinyos 4096 Jan  3 11:37 ..
-rw-r--r-- 1 tinyos tinyos    0 Jan  3 10:01 access.log
-rw-r--r-- 1 tinyos tinyos    0 Jan  3 10:01 sys.log

[download]

In some of the files I added the IP that you are searching and also some dummy text (incident error report). Sample of one file bellow:

$ cat 2018-01-03/access.log 
127.0.0.1 This is insident 1 in 2018-01-03
127.0.0.2 This is insident 2 in 2018-01-03
127.0.0.3 This is insident 3 in 2018-01-03
127.0.0.1 This is second insident 4 in 2018-01-03

[download]

If I understand correctly from your description something like that should do what you need. If not it should be close to 95% minor modifications to bring it close to your desired output.

Hope this helps, BR.

Seeking for Perl wisdom...on the process of learning...not there...yet!

In reply to Re: Grep logs by start date and end date in different directories by thanos1983
in thread Grep logs by start date and end date in different directories by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.