SSH and checking log files

naChoZ has asked for the wisdom of the Perl Monks concerning the following question:

I've been trying to noodle through a good way of doing this. I have a script that can use STDIN to parse a maillog file, so I can run ssh server.example.com cat /var/log/maillog | ./maillog_parse.pl --import and it works just swell.

Now I would like to incrementally check the log. So, how would one tell and seek over an ssh connection? I'm open to other suggestions, but keep in mind that some of these logs are huge. As long as it can perform checks periodically, factor in log rotation, and not rely on the timestamp of a log line, that's all I need.

--
"A long habit of not thinking a thing wrong, gives it a superficial appearance of being right." -- Thomas Paine
naChoZ

Comment on SSH and checking log files Download Code

Replies are listed 'Best First'.
Re: SSH and checking log files by Aristotle (Chancellor) on Aug 10, 2004 at 19:39 UTC
You can't. You'd have to do something with `tail` or such on the remote end. Makeshifts last the longest.	[reply]
Re^2: SSH and checking log files by naChoZ (Curate) on Aug 10, 2004 at 23:04 UTC
Ok. I thought it might be something like that, but I thought I'd ask in case there was IO::Handle magic I didn't know about or something along those lines. -- "A long habit of not thinking a thing wrong, gives it a superficial appearance of being right." -- Thomas Paine naChoZ	[reply]
Re: SSH and checking log files by pzbagel (Chaplain) on Aug 10, 2004 at 19:49 UTC
So, how would one tell and seek over an ssh connection? You can't. The realistic solution is to write a script that can replace the 'cat' in your command-line with a command that will print the "new" log lines that you want to see. There is probably a myriad of ways to do that. Also, using a second script it will be trivial to take log-rotation and other factors into account since you will be on the local system and could check that type of stuff easily.	[reply]
Re: SSH and checking log files by roju (Friar) on Aug 11, 2004 at 00:38 UTC
You could always tunnel nfs over ssh ;)	[reply]
Re: SSH and checking log files by rsteinke (Scribe) on Aug 11, 2004 at 06:33 UTC
Another possibilty is replacing your cat command with perl -e 'some one liner'. This gets a bit tricky, since you have to double escape the script text (once to pass through the local shell, and once to pass through the remote one). Using Net::SSH to internalize the ssh call to your local perl script helps this a bit. The main drawback is that starting a perl interpreter on the remote system is significantly more expensive than running 'cat' or 'tail'. Ron Steinke <rsteinke@w-link.net>	[reply]
Re: SSH and checking log files by ikegami (Patriarch) on Aug 10, 2004 at 20:45 UTC
How about something like: `ssh server.example.com \ cat_partial.pl /var/log/maillog 62548 \ \| ./maillog_parse.pl` [download] You could write a small perl script (cat_partial.pl) that will execute on the server, doing your seeks and rotation checks based on command line arguments. (The example above would seek to 62548.)	[reply] [d/l]