Messy apache error logs got you down? Squinting so hard trying to look through all the repeated information that's not useful that you're leaving visible marks on your eyeballs? No longer. Use err_watch and watching your error log will be more fun than TV! It helps you strip out all the redundant information and shows only what is left: relevent things. Run with -h for switching options. Features requests appreciated.
#!/usr/bin/perl -w # usage information found at the bottom or by using -h use strict; use Term::ReadKey; use Time::Local; use Cwd qw(abs_path); $|++; my (%Options, $wchar, $hchar); sub get_options; sub printNice($); sub mon2num($); sub timelocal_str; sub bufferstr; main: { %Options = get_options(); my $ph; if( $Options{watch_file} && $Options{watch_file} ne '-' ) { + eval { open LOG, qq{tail -n0 -f "$Options{watch_file}"|} } or +die "Couldn't open pipe to tail: $!"; local $SIG{PIPE} = sub { die "tail pipe broke" }; $ph = \*LOG; } else { $ph = \*STDIN; } my $cur; my $lastcur = 0; while( <$ph> ) { chomp; if( /^\[([^\]]+)\]/ ) { ($wchar, $hchar, ) # $wpixels, $hpixels) = GetTerminalSize(); my $lt = timelocal_str($1); if( $lt >= $cur + $Options{group_sec}) { $lastcur = $cur = $lt; print bufferstr() . localtime($lt) . ":\n"; } if( $Options{show_times} && $lt != $lastcur ) { $lastcur = $lt; print( (' 'x($Options{indent} - 4)) . localtime($lt) . + "\n"); } s/^\[[^\]]+\]//; s/\s*\[error\]\s*// if( $Options{error} ); s/\s*\[client [^\]]+\]\s*// if( $Options{client} ); s/, referer: .*// if( $Options{referer} ); foreach my $str ( @{ $Options{dir_filter} } ) { s/$str//g; } printNice "$_\n"; } } close LOG; } sub printNice($) { my $str = shift; print join('', ' 'x($Options{indent})) . '* ' . substr($str, 0, $w +char - $Options{indent} -3, ''); while($str ne '') { print "\n" . join('', ' 'x($Options{indent}+2)) . substr($str, + 0, $wchar - $Options{indent} - 2 -1, ''); } } sub mon2num($) { my $month = shift; my %h = qw( jan 0 feb 1 mar 2 apr 3 may 4 jun 5 jul 6 aug 7 sep 8 oct 9 nov 10 dec 11 ); return $h{ lc substr($month, 0, 3) }; } sub timelocal_str { #does the opposite of localtime, my $seconds = tim +elocal( localtime().'' ) my ($str) = @_; # Sat Jan 21 01:16:50 2006 $str =~ s/\s+|\s+/ /g; $str =~ s/^\s+/ /g; $str =~ s/\s+$/ / +g; my ($wd, $mon, $mday, $hr, $min, $sec, $yr) = split /[ \:]/, $str; return timelocal($sec, $min, $hr, $mday, mon2num $mon, $yr); } sub bufferstr { my $buffer = ''; if( $Options{buffer} ) { my $buffer_line = ''; while( length($buffer_line) + length($Options{buffer}) < $wcha +r ) { $buffer_line .= $Options{buffer}; } $buffer .= $buffer_line; } $buffer .="\n"x($hchar + 1); return $buffer; } sub get_options { my %options = ( 'referer' => 1, 'client' => 1, 'error' => 1, 'dir_filter' => [], 'indent' => 6, 'group_sec' => 2, 'show_times' => undef, 'watch_file' => q{-}, 'buffer' => q{ == - == -}, ); while( $_ = shift @ARGV ) { my $had_unshift = 0; if( /^-[a-z0-9]{2,}/i ) { my $param = substr($_, 0, 2, ''); my ($p, $v) = split /=/; unshift @ARGV, $v if( $v ); foreach my $n ( reverse split //, $p ) { unshift @ARGV, "-$n"; } $_ = $param; } elsif( /=/ ){ my ($k, $v) = split /=/, $_, 2 ; $_ = $k; unshift @ARGV, $v; $had_unshift = 1; } if( /--help|-h/ ) { print usage(); exit(1); } elsif( /--referer|-r/ ) { $options{referer} = 0; } elsif( /--client|-c/ ) { $options{client} = 0; } elsif( /--error|-e/ ) { $options{error} = 0; } elsif( /--dir|-d/ ) { my $d = shift(@ARGV) || '.' ; my $rel_d; eval { $rel_d = abs_path($d) }; #bad paths are OK. $d = $rel_d if $rel_d; push @{ $options{dir_filter} }, $d ; } elsif( /--indent|-i/ ) { $options{indent} = shift(@ARGV); } elsif( /--group-sec|-t/ ) { $options{group_sec} = shift(@ARGV); } elsif( /--buffer|-B/ ) { $options{buffer} = shift(@ARGV); } elsif( /--show-times|-s/ ) { $options{show_times} = 1; } else { $options{watch_file} = $_; } } return %options; } sub usage { return q{ err_watch [-rce] [-d=dir]* [-i=n] [-t=sec] [-B=str] [watched_file] -r,--referer Leave the referer line in the log -c,--client Leave the [client ...] info in the log -e,--error Leave the [error] info in the log -d=dir,--dir Filter out the string matching dir or the current directory (filename collap +sing), this option may be given multiple times -i=n,--indent Indent the contents of the watched file by +n spaces -t=sec,--group-sec Group errors in sec seconds icrements -B=str,--buffer The string defining your "group buffer stri +ng" it will print 1 row of these repating all + the way across. -s,--show-times if set, it will print a timestamp starting +wherever the logged timestamp changes, it will sti +ll group them according to -t however }; }

Replies are listed 'Best First'.
Re: err_watch for when you're errors are growing like hair or...
by qbxk (Friar) on Jan 22, 2006 at 03:10 UTC
    ..and some things I didn't want to say in the description:

    I only worked with this one machine when I developed it, I don't know if all apache logs look like this expects them to, and I didn't feel like checking, let me know if this doesn't work with your log for some reason, and post a copy of the part that sticks out.

    I know, I wrote my own command line parser... I didn't like any of the GetOpts, and well, this is pretty clean, you'll have to admit... please let me know if you find a bug in parsing your command line.


    It's not what you look like, when you're doin' what you’re doin'.
    It's what you’re doin' when you’re doin' what you look like you’re doin'!
         - Charles Wright & the Watts 103rd Street Rhythm Band
Re: err_watch for when your errors are growing like hair or...
by chanio (Priest) on Jan 22, 2006 at 16:55 UTC
    It would be nice if you could parametrize data to: adapt to other log formats, and to other locales (just a different .ini file).

    Use Term::ANSIColor for color coding. A color shift says more than 1K. words!

    Perhaps, using this code, if you only need the days of the week locale...

    perl -MPOSIX -Mlocale -e' ## GET LOCAL DATES AND NAMES my ($tnow,$gday)=(time,int(60*60*24)); ## TODAY IN SECONDS, SECONDS I +N ONE DAY (4 ADDING) my %wd; $wd{(localtime($tnow+($gday*$_)))[6]}=strftime("%a",localtime(($gday*$ +_)+$tnow)) foreach (0..6); ## RELATE +NUM. OF DAY OF THE WEEK WITH ITS NAME print("$wd{$_}, ") foreach (sort(keys(%wd))); ## DISPLAY NAMES IN ORDE +R (SEE LAST LINE) print $/ ## (WEEK DAYS IN SPANISH) __END__ dom, lun, mar, mié, jue, vie, sáb, '