in reply to Re: How to grep matching IP address from a log file?
in thread How to grep matching IP address from a log file?

Hello, thank you so much for your reply.

As you have mentioned, there will be 2 Ip addresses per line. One of it is my source IP, the other is the IP addresses I want to extract.

These are the fields of the actual line:

date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c +-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken

Sample logs

2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=66&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 530 2017-12-08 07:01:39 <s-ip> GET /course-listing.aspx - 443 - <c-ip> cur +l/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2 +.3+libidn/1.18+libssh2/1.4.2 200 0 0 140 2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=24&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 93 2017-12-08 07:01:40 <s-ip> GET /logistics.aspx - 443 - <c-ip> curl/7.1 +9.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2.3+li +bidn/1.18+libssh2/1.4.2 200 0 0 46 2017-12-08 07:01:40 <s-ip> GET /course-detail.aspx id=23&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 140

c-ip being the IP address I want to extract

Replies are listed 'Best First'.
Re^3: How to grep matching IP address from a log file?
by haukex (Archbishop) on Dec 19, 2017 at 13:18 UTC

    hippo's reply hopefully makes clear: the sample input you have shown appears to still not really be representative of real-world data, because I assume there is a chance that the "s-port" may not always be 443, and the user agent is unlikely to always start with curl, etc. Sorry, but what you have told us so far is still not enough information to build a robust parser. For example, if you assumed that the regex I showed will always match twice per line, and then your log file happens to contain an IP address in the cs-uri-stem cs-uri-query fields, and/or a hostname in one of the s-ip c-ip fields, code based on this assumption will break.

    The way I would suggest approaching this is to see if you can find out more specifications about the fields - in particular, whether any of the date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip fields may contain whitespace, and if they can, how they might be delimited, etc. If, for example, the specification says that none of these fields may contain whitespace, it may turn out that LanX's suggestion of split may be enough. But if not, you'll have to write a regex to parse the line.

Re^3: How to grep matching IP address from a log file?
by hippo (Archbishop) on Dec 19, 2017 at 09:02 UTC
    c-ip being the IP address I want to extract

    Since you haven't put this actual address in, it cannot be searched for in the sample data. But that's OK because really you just want whatever is in its place, so this will suffice:

    #!/usr/bin/env perl use strict; use warnings; while (<DATA>) { my ($ip) = (/ 443 - (.+) curl/); print "Line $.: Found $ip\n"; } __DATA__ 2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=66&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 530 2017-12-08 07:01:39 <s-ip> GET /course-listing.aspx - 443 - <c-ip> cur +l/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2 +.3+libidn/1.18+libssh2/1.4.2 200 0 0 140 2017-12-08 07:01:39 <s-ip> GET /course-detail.aspx id=24&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 93 2017-12-08 07:01:40 <s-ip> GET /logistics.aspx - 443 - <c-ip> curl/7.1 +9.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3.27.1+zlib/1.2.3+li +bidn/1.18+libssh2/1.4.2 200 0 0 46 2017-12-08 07:01:40 <s-ip> GET /course-detail.aspx id=23&catColor=0 44 +3 - <c-ip> curl/7.19.7+(x86_64-redhat-linux-gnu)+libcurl/7.19.7+NSS/3 +.27.1+zlib/1.2.3+libidn/1.18+libssh2/1.4.2 200 0 0 140