in reply to How to grep matching IP address from a log file?

You could use Regexp::Common::net - this will extract any IP's from the line:

use warnings; use strict; use Regexp::Common qw/net/; my $regex = qr{ (?<ip> \b $RE{net}{IPv4} \b ) }msx; my $filename = 'mylogFile.log'; open my $fh, '<:encoding(UTF-8)', $filename or die "$filename: $!"; while (my $line = <$fh>) { while ($line=~/$regex/g) { print "<", $+{ip}, ">\n"; } } close $fh;

However, in your sample input you wrote <my source IP>. If this is an actual IP address, then that will be matched too. If you don't want that, you need to extend the regex to match the surrounding parts of the line. Also, if that's the case, you haven't shown what the actual lines look like, i.e. your input isn't representative, so we can't really help there. All I can say is that if you only want to match one IP per line, you can replace the inner while loop with a single if. Have a look at this for some advice on designing regexes.

In the code you posted, you are reading the entire file into an array, which is a bit wasteful, and it'd probably be better if you used a regular while (<$fh>) loop instead, like what I showed above and as is explained in e.g. Files and I/O and I/O Operators. Also, note that with your current regex, you're just matching the first digits of an IP address, and you may get false positives.

Replies are listed 'Best First'.
Re^2: How to grep matching IP address from a log file?
by dotowwxo (Acolyte) on Dec 19, 2017 at 08:19 UTC

    Hello, thank you so much for your reply.

    As you have mentioned, there will be 2 Ip addresses per line. One of it is my source IP, the other is the IP addresses I want to extract.

    These are the fields of the actual line:

    date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c +-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken

    Sample logs

    2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=66&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 530 2017-12-08 07:01:39 <s-ip> GET /course-listing.aspx - 443 - <c-ip> cur +l/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2 +.3+libidn/1.18+libssh2/1.4.2 200 0 0 140 2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=24&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 93 2017-12-08 07:01:40 <s-ip> GET /logistics.aspx - 443 - <c-ip> curl/7.1 +9.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2.3+li +bidn/1.18+libssh2/1.4.2 200 0 0 46 2017-12-08 07:01:40 <s-ip> GET /course-detail.aspx id=23&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 140

    c-ip being the IP address I want to extract

      hippo's reply hopefully makes clear: the sample input you have shown appears to still not really be representative of real-world data, because I assume there is a chance that the "s-port" may not always be 443, and the user agent is unlikely to always start with curl, etc. Sorry, but what you have told us so far is still not enough information to build a robust parser. For example, if you assumed that the regex I showed will always match twice per line, and then your log file happens to contain an IP address in the cs-uri-stem cs-uri-query fields, and/or a hostname in one of the s-ip c-ip fields, code based on this assumption will break.

      The way I would suggest approaching this is to see if you can find out more specifications about the fields - in particular, whether any of the date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip fields may contain whitespace, and if they can, how they might be delimited, etc. If, for example, the specification says that none of these fields may contain whitespace, it may turn out that LanX's suggestion of split may be enough. But if not, you'll have to write a regex to parse the line.

      c-ip being the IP address I want to extract

      Since you haven't put this actual address in, it cannot be searched for in the sample data. But that's OK because really you just want whatever is in its place, so this will suffice:

      #!/usr/bin/env perl use strict; use warnings; while (<DATA>) { my ($ip) = (/ 443 - (.+) curl/); print "Line $.: Found $ip\n"; } __DATA__ 2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=66&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 530 2017-12-08 07:01:39 <s-ip> GET /course-listing.aspx - 443 - <c-ip> cur +l/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2 +.3+libidn/1.18+libssh2/1.4.2 200 0 0 140 2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=24&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 93 2017-12-08 07:01:40 <s-ip> GET /logistics.aspx - 443 - <c-ip> curl/7.1 +9.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2.3+li +bidn/1.18+libssh2/1.4.2 200 0 0 46 2017-12-08 07:01:40 <s-ip> GET /course-detail.aspx id=23&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 140