chris2013 has asked for the wisdom of the Perl Monks concerning the following question:

Dear All,

I'd like to parse a Charon (StrongSwan) logfile.

Somewhere it logs a failed login. If this is found, I'd like to lookup the username and IP address, that are mentioned some lines above.

What's the best approach? Is there any framework that can do that? Where I only have to fill-in the regex?

Should I read the whole logfile to an array and then go back? Or is there something like grep? I've tried to use a shell script and let perl one-liners do the regex stuff. Unfortunately I couldn't escape the single quotes in front and after the e-mail address. They have to be quoted because of the shell and the regex. Tried to write them to an evironment variable but that didn't work. Probably, there are completely different and better solutions. 

export MYP="identity '([A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,})'"
EMAIL=`grep -B15 "$DATE" $LOGFILE | grep "$CONID" | grep identity | pe
+rl -nle 'print "$1" if /$ENV{MYP}/'`

[download]

Example: 

Apr 29 10:01:10 vpn.example.com charon: 07[NET] <con1|33> received pac
+ket: from 2.204.0.13[55842] to 27.3.213.112[4500] (80 bytes)
Apr 29 10:01:10 vpn.example.com charon: 14[IKE] <con1|33> received EAP
+ identity 'jd@example.com'
Apr 29 10:01:11 vpn.example.com charon: 14[IKE] <con1|33> EAP-MS-CHAPv
+2 verification failed, retry (1)

[download]

- Chris

Replies are listed 'Best First'.
Re: Parse multiline logfile
by BrowserUk (Patriarch) on Apr 29, 2016 at 21:25 UTC
    Should I read the whole logfile to an array and then go back?

    Log files tend to get very large, so reading the whole lot might cause problems.

    The following assumes that the relevant lines will be grouped together and only keeps 4 lines in memory at a time; change the constant if you need more.

    It then runs the regex against the concatenation of the current 4 lines and outputs the relevant fields when a match is found.

    It works on a test file consisting of a few dozen repetitions of your posted data with most of the sets of lines modified not to match; beyond that testing depends upon the nature of your data:

    perl -nwle" if(push( @s, $_ ) > 4 ){ shift @s } qq[@s]=~m[.+from (\S+)
+.+identity ('[^']+').+verification failed]s and print qq[$1:$2]" junk
+.dat

    [download]

    Remember to change/adjust/escape the "s & 's for *nix.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
Re: Parse multiline logfile
by choroba (Cardinal) on Apr 29, 2016 at 20:50 UTC
    How do you identify that a failure corresponds to an identity? It might be possible to have several people logging in at the same time, so many lines with "received", and then a failure. How would you pick the corresponding identity? E.g. is the con1|33 unique?

    ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord
}map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

    [download]
[reply]
[d/l]
[select]
      Thank you for your replies. I could modify my script with your simpler regex so that it matches now. If the con1|33 is unique, is a good question. The strongswan guys haven't replied yet. At least, this ID appears only for one login at a time. I don't know when it is reset. - Chris
[reply]

Back to Seekers of Perl Wisdom