Re^2: Undefined Error

Replies are listed 'Best First'.
Re^3: Undefined Error by poj (Abbot) on Jun 15, 2013 at 12:33 UTC
Can you explain "the code is not working" ?. The missing SRCH=Q means you will not get a line like this in the results with the code and data shown. `User xyz123 had 1 searches on connection 13573` [download] poj	[reply] [d/l]
Re^3: Undefined Error by hdb (Monsignor) on Jun 15, 2013 at 13:33 UTC
What do you expect as output for your dataset? I thought the following would be fine: `User xyz123 had 1 searches on connection 13570 User xyz123 had 1 searches on connection 13572 User someoneelse had 1 searches on connection 13571` [download]	[reply] [d/l]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 15, 2013 at 14:01 UTC
Well, I want to print "bad User" if count is greater than 3, that is the only output I want. `print "\t=> Bad user!\n" if @{$searches{$conn}} > 3;` [download]	[reply] [d/l]
Re^5: Undefined Error by hdb (Monsignor) on Jun 15, 2013 at 14:05 UTC
In your dataset, there is no one with more than 3 searches. If you do not like the other output, remove the print statements. If you want to check for the total number of searches, you need to sum up across connections like this: `for my $user (keys %users) { my $totalsearches=0; for my $conn (@{$users{$user}}) { if( exists $searches{$conn} ) { print "User $user had ".scalar( @{$searches{$conn}} )." searches + on connection $conn\n"; $totalsearches += @{$searches{$conn}}; } } print "User $user had $totalsearches searches in total\n"; print "\t=> Bad user!\n" if $totalsearches > 3; }` [download]	[reply] [d/l]
Re^5: Undefined Error by poj (Abbot) on Jun 15, 2013 at 14:29 UTC
If you only want a total count the code can be simplified ; `use strict; use warnings; my %users; my %conn; while (<DATA>) { # I use DATA handle instead of $fh for convenience if( /BIND/ ) { my( $conn, $uid ) = /conn=(\d+).uid=(.?),/; $conn{$conn} = $uid; } if( /SRCH=Q/ ) { my ($timestamp, $conn) = /\[(.*?)\] conn=(\d+)/; my $uid = $conn{$conn}; ++$users{$uid}; } } for my $uid (keys %users) { my $count = $users{$uid}; print "\t=> Bad user $uid ! count = $count\n" if $count>3; }` [download] poj	[reply] [d/l]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 15, 2013 at 15:22 UTC
Actually, there is some issue with regex, where I am putting /^BIND$/ to match exact word, as there is another string "UNBIND". is this not the right way to match exact string /^BIND$/? Please let me know if it is not. Thanks,	[reply]
Re^5: Undefined Error by poj (Abbot) on Jun 15, 2013 at 15:46 UTC
`/^BIND$/` matches if that is the only word on the line. Try adding word boundaries \b like this ; `while (<DATA>){ if (/\bBIND\b/){ print $_; } } __DATA__ this is a BIND sentence I have UNBIND in this one BIND as first word last word is BIND` [download] poj	[reply] [d/l] [select]
Re^5: Undefined Error by hdb (Monsignor) on Jun 15, 2013 at 15:46 UTC
BIND seems to be surrounded by spaces, so `/\sBIND\s/` should work. Your suggestion would only fit a string (or line) containing BIND and nothing else as the caret is signifying the start of the string (or line) and dollar the end of string (or line).	[reply] [d/l] [select]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 15, 2013 at 16:22 UTC
Thank You, both works ( /b and /s). One last query related to this post, I would like to add time search, where the time has following format : `[04/Jun/2013:15:06:13` [download] I would like to scan the log for last one hour only whenever I run the script. Thanks for all your effort.	[reply] [d/l]
Re^5: Undefined Error by poj (Abbot) on Jun 15, 2013 at 17:13 UTC
Remove the extra print statements if you are happy it works as expected. use strict; use warnings; ## add this use DateTime::Format::Strptime qw(strptime); my $NOW = DateTime->now(); my $MAX_AGE = 60; print "Time now is $NOW\n"; ## my %users; my %conn; while (<DATA>) { # I use DATA handle instead of $fh for convenience if( /\bBIND\b/ ) { my( $conn, $uid ) = /conn=(\d+).uid=(.?),/; $conn{$conn} = $uid; } if( /SRCH=Q/ ) { my ($timestamp, $conn) = /\[(.*?)\] conn=(\d+)/; ## add this my $dt = strptime('%d/%B/%Y:%T',substr($timestamp,0,20)); my $age = $dt->delta_ms( $NOW )->in_units('minutes'); print $timestamp." = " .$dt->datetime." ; $age mins\n"; ## ## add condition if ($age >= $MAX_AGE){ my $uid = $conn{$conn}; ++$users{$uid}; } } } for my $uid (keys %users) { my $count = $users{$uid}; print "\t=> Bad user $uid ! count = $count\n" if $count > 3; } [download] poj	[reply] [d/l]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 15, 2013 at 18:02 UTC
Thanks for your response. I do not have "DateTime::Format::Strptime" installed, and it is not possible to have the module installed. Any other alternative?	[reply]
Re^5: Undefined Error by poj (Abbot) on Jun 15, 2013 at 19:07 UTC
Not ideal (not exactly 60 mins 0 seconds and won't work either side of midnight) but try this #!perl use strict; use warnings; ## add this my $MAX_AGE = 60; # minutes # calc cut off date time my @mth = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec); my @f = localtime(); my $TODAY = sprintf "%02d/%s/%4d",$f[3],$mth[$f[4]],$f[5]+1900; my $START_MINUTE = $f[2]60+$f[1] - $MAX_AGE; print "Now is $TODAY $START_MINUTE mins\n"; ## my %users; my %conn; while (<DATA>) { # I use DATA handle instead of $fh for convenience if( /\bBIND\b/ ) { my( $conn, $uid ) = /conn=(\d+).uid=(.?),/; $conn{$conn} = $uid; } if( /SRCH=Q/ ) { my ($timestamp, $conn) = /\[(.?)\] conn=(\d+)/; ## add this my ($date,$h,$m,undef) = split ':',$timestamp,4; next unless ($date eq $TODAY); my $minutes = $h*60 + $m; print $timestamp." = $date $h $m ; $minutes <=> $START_MINUTE\n"; ## # add condition if ($minutes >= $START_MINUTE){ my $uid = $conn{$conn}; ++$users{$uid}; } } } for my $uid (keys %users) { my $count = $users{$uid}; print "\t=> Bad user $uid ! count = $count\n" if $count > 3; } [download] poj	[reply] [d/l]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 16, 2013 at 07:05 UTC
Hi Poj, I see that you've changed the logic, can you tell me know how to just disable the time part, so that I can run the script to verify the main logic. Thanks,	[reply]
Re^5: Undefined Error by poj (Abbot) on Jun 16, 2013 at 07:27 UTC
OK, comment out this line ; `## next unless ($date eq $TODAY);` [download] and the if `# add condition ## if ($minutes >= $START_MINUTE){ my $uid = $conn{$conn}; ++$users{$uid}; ## }` [download] poj	[reply] [d/l] [select]
Re^5: Undefined Error by johnrein55 (Novice) on Jun 16, 2013 at 19:21 UTC
Hi Poj, I tried running the code by commenting the rows as suggested, however I do not get the result ( ie bad user) Here is the log, can you try from your side, you must see bad user getting printed. [04/Jun/2013:13:06:13 -0600] conn=13570 op=14 msgId=13 - BIND dn="uid= +xyz123,ou=People,o=xyz.com" method=128 version=3 [04/Jun/2013:15:06:13 -0600] conn=13570 op=14 msgId=15 - RESULT err=0 +tag=101 nentries=48030 etime=139 SRCH=Q [04/Jun/2013:13:06:13 -0600] conn=13572 op=14 msgId=13 - BIND dn="uid= +xyz123,ou=People,o=xyz.com" method=128 version=3 [04/Jun/2013:15:06:13 -0600] conn=13572 op=14 msgId=15 - RESULT err=0 +tag=101 nentries=48030 etime=139 SRCH=Q [04/Jun/2013:17:06:13 -0600] conn=13571 op=14 msgId=13 - BIND dn="uid= +someoneelse,ou=People,o=xyz.com" method=128 version=3 [04/Jun/2013:18:06:17 -0600] conn=13571 op=14 msgId=15 - RESULT err=0 +tag=101 nentries=48030 etime=139 SRCH=Q [04/Jun/2013:13:06:13 -0600] conn=13573 op=14 msgId=13 - BIND dn="uid= +xyz123,ou=People,o=xyz.com" method=128 version=3 [04/Jun/2013:15:06:13 -0600] conn=13573 op=14 msgId=15 - RESULT err=0 +tag=101 nentries=48030 etime=139 SRCH=Q [04/Jun/2013:13:06:13 -0600] conn=13574 op=14 msgId=13 - BIND dn="uid= +xyz123,ou=People,o=xyz.com" method=128 version=3 [04/Jun/2013:15:06:13 -0600] conn=13574 op=14 msgId=15 - RESULT err=0 +tag=101 nentries=48030 etime=139 SRCH=Q [04/Jun/2013:13:06:13 -0600] conn=13575 op=14 msgId=13 - BIND dn="uid= +someone,ou=People,o=xyz.com" method=128 version=3 [04/Jun/2013:15:06:13 -0600] conn=13575 op=14 msgId=15 - RESULT err=0 +tag=101 nentries=48030 etime=139 [download]	[reply] [d/l]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 17, 2013 at 13:50 UTC
Hi Poj, I am only getting print statement for time, here is the output, i should get number of counts, and print statement as bad user. `Now is 17/Jun/2013 404 mins 17/Jun/2013:06:12:37 -0600 = 17/Jun/2013 06 12 ; 372 <=> 404 17/Jun/2013:06:32:34 -0600 = 17/Jun/2013 06 32 ; 392 <=> 404 17/Jun/2013:06:32:46 -0600 = 17/Jun/2013 06 32 ; 392 <=> 404 17/Jun/2013:06:32:55 -0600 = 17/Jun/2013 06 32 ; 392 <=> 404 17/Jun/2013:06:33:28 -0600 = 17/Jun/2013 06 33 ; 393 <=> 404` [download] Thanks.	[reply] [d/l]
Re^5: Undefined Error by poj (Abbot) on Jun 17, 2013 at 14:24 UTC
See what you get with the count condition commented out like this ; `print "\t=> Bad user $uid ! count = $count\n";# if $count > 3;` Or change the value from 3 to 0 poj	[reply] [d/l]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 17, 2013 at 19:46 UTC
Hi Poj, I regret, the script does print count, however the issue is that there are lot of logs, is there any way to put the hashes within file db, so that it is not consuming memory? Thanks.	[reply]
Re^5: Undefined Error by poj (Abbot) on Jun 17, 2013 at 20:02 UTC
I doubt if memory usage is your problem. How many is "a lot" and how big are the logs, i.e. how many lines in the biggest ? poj	[reply]
Re^4: Undefined Error by johnrein55 (Novice) on Jun 17, 2013 at 20:07 UTC
I took the prod log, an hour log is of 200 MBs. Is that low/high? To be on safer side, I would like to dump the search onto the disk, and then perform the searches ( ie bind etc) what do you suggest?	[reply]
Re^5: Undefined Error by poj (Abbot) on Jun 18, 2013 at 16:05 UTC
Not quite sure what you mean but assuming you have a number of logs all in one folder, then this script will scan them and extract only the records you are interested in into a single file which you can process with the 'bad user' script. #!perl use strict; # start time my $t0 = time(); # input - create list of files my @files = grep { /.log/ } glob("*") ; # output my $count_out = 0; my $outfile = 'output.log'; open OUT,'>',$outfile or die "Could not open $outfile : $!"; # process for my $infile (@files){ next if ($infile eq $outfile); open IN,'<',$infile or die "Could not open $infile : $!"; print "Reading $infile .. "; my $count_in = 0; while (<IN>){ ++$count_in; if (/BIND\|SRCH=Q/){ print OUT; ++$count_out; } } close IN; print "$count_in records read\n"; } # finish close OUT; my $dur = time() - $t0; print "Finished in $dur secs \n $count_out written to $outfile\n"; [download] poj	[reply] [d/l]
Re^5: Undefined Error by johnrein55 (Novice) on Jun 19, 2013 at 15:10 UTC
Thanks Poj. Actually, my intent is to dump the content of hash into disk instead of memory. Anyway, I will try to utilize the provide script. However, can you put hourly logic into this as you had before?	[reply]