Re^2: Best practice ipc and Log::Log4perl

What I want:

read unbuffered from a child process ( long running process ).

capture childprocess stderr & stdout separately

write the childprocess stdout and stderr stream to a logfile and tag lines with a <timestamp> and a tag <INFO> or <ERROR> depending on the source stream.

IPC::Run3 almost does what I need.

#!/usr/bin/perl
# timestamp.pl
$|++;
while(time - $^T<10){
  select undef,undef,undef, 0.1;
  if ( rand()<0.5) {
    print STDERR  time(),  " warn\n";
  } else {
    print time(), " info\n";
  }
}
__END__
[download]

use IPC::Run3 qw(run3);
my @cmd = ('perl', 'timestamp.pl');
run3 \@cmd, undef, \&out, \&err;
sub out {
  print time(), " out: $_\n";
}
sub err {
  print time(), " err: $_\n";
}
__END__
# The problem is that run3 buffers the output.
1215162770 out: 1215162760 info
1215162770 out: 1215162760 info
1215162770 out: 1215162761 info
1215162770 out: 1215162761 info
1215162770 out: 1215162761 info
1215162770 out: 1215162761 info
1215162770 out: 1215162762 info
1215162770 out: 1215162762 info
1215162770 out: 1215162762 info
1215162770 out: 1215162763 info
[download]

print+qq(\L@{[ref\&@]}@{['@'x7^'!#2/"!4']});

Comment on Re^2: Best practice ipc and Log::Log4perl Select or Download Code

Replies are listed 'Best First'.
Re^3: Best practice ipc and Log::Log4perl by starbolin (Hermit) on Jul 05, 2008 at 03:00 UTC
Run3 apparently doesn't do concurrency. IPC:Run can be made to do what you want. This works: #!/usr/bin/perl # timestamp.pl while(time - $^T<10){ select undef,undef,undef, 0.1; if ( rand()<0.5) { print STDERR time(), " warn\n"; } else { print time(), " info\n"; } } print "end\n"; __END__ use IPC::Run qw ( start pump finish timeout ); my $in; my $out; my @cmd = qw( perl timestamp.pl); ## Build the harness, open all pipes, and launch the subprocesses my $h = start \@cmd, \$in, \$out, \$out ; $in = "" ; ## Now do I/O. start() does no I/O. pump $h while length $in ; ## Wait for all input to go ## Now do some more I/O. while (1){ pump $h until $out =~ s/(.+)\n$//gm ; last if ("end" eq $1); print time, " : $1\n"; } ## Clean up finish $h or die "cat returned $?" ; __END__ 1215226647 : 1215226647 info 1215226647 : 1215226647 info 1215226647 : 1215226647 warn 1215226647 : 1215226647 info 1215226647 : 1215226647 warn 1215226647 : 1215226647 warn 1215226647 : 1215226647 info 1215226647 : 1215226647 info 1215226647 : 1215226647 info 1215226648 : 1215226648 warn [download] s//----->\t/;$~="JAPH";s//\r<$~~/;{s\|~$~-\|-~$~\|\|\|s \|-$~~\|$~~-\|\|\|s,<$~~,<~$~,,s,~$~>,$~~>,, $\|=1,select$,,$,,$,,1e-1;print;redo}	[reply] [d/l]
Re^4: Best practice ipc and Log::Log4perl by codeacrobat (Chaplain) on Jul 05, 2008 at 08:41 UTC
Thanks for your solution IPC::Run is probably the best way to go. If you set $\|++ in timestamp.pl we have it. perl -MIPC::Run=run -e ' run( [ "perl", "timestamp.pl" ], "1>", sub { print "out " . time() . " " , @_ }, "2>", sub { print "err ". time() . " " , @_} ) ' out 1215247029 1215247029 info err 1215247029 1215247029 warn err 1215247029 1215247029 warn out 1215247029 1215247029 info err 1215247029 1215247029 warn err 1215247029 1215247029 warn out 1215247030 1215247030 info err 1215247030 1215247030 warn err 1215247030 1215247030 warn err 1215247030 1215247030 warn err 1215247030 1215247030 warn out 1215247030 1215247030 info err 1215247030 1215247030 warn out 1215247030 1215247030 info out 1215247030 1215247030 info out 1215247031 1215247030 info out 1215247031 1215247031 info err 1215247031 1215247031 warn err 1215247031 1215247031 warn out 1215247031 1215247031 info err 1215247031 1215247031 warn out 1215247031 1215247031 info out 1215247031 1215247031 info err 1215247031 1215247031 warn err 1215247031 1215247031 warn [download] `print+qq(\L@{[ref\&@]}@{['@'x7^'!#2/"!4']});`	[reply] [d/l] [select]
Re^5: Best practice ipc and Log::Log4perl by jointwork (Initiate) on Apr 25, 2009 at 20:04 UTC
Hi, Excuse me for posting to this old threat, but I run in a similar situation where IPC::Run and Log::Log4perl were combined. I must remark that the title is actually misleading because no one has mentioned log4perl in his/her solution. Thus I will: `use IPC::Run qw(run new_chunker); # snip-snap-snip # ... my @cmd = qw(find /usr/bin -type f); run(\@cmd, "1>", new_chunker, \&logSdtout, "2>", new_chunker, \&logSdt +err); sub logSdtout { $logger->info(@_); } sub logSdterr { $logger->error(@_); }` [download] edit: you need to pass the filter "new_chunker" to IPC::Run to break the output from your external program into newlines. Otherwise you will run into strange exceptions (undefined values) causing IPC::Run to fail. Cheers, Daniel	[reply] [d/l]