pipe issue (linux)

leostereo has asked for the wisdom of the Perl Monks concerning the following question:

Hi guys,
I was trying to pipe two perl scripts
I have a data provider and a data process scritp.
First the data provider is
dhcp_line_dispacher.pl

#!/usr/bin/perl -w
use strict;
use File::Tail;

my $file = File::Tail->new(
        name =>'/var/log/messages', 
        interval => 1, 
        maxinterval => 1, 
#        resetafter=> 5, 
        );

while (defined(my $line=$file->read)) {
 
    print $line;

}
[download]

and , line_process.pl is:

#!/usr/bin/perl -w
use strict;
use Net::Ping;
my $result;

while (<>){

                my $p = Net::Ping->new('icmp',3);

                if ($_ =~ /^(.*) dns02cor dhcpd: DHCPACK on ([0-9\.]+)
+ to ([[:xdigit:]:]+).* ([0-9\.]+)/) {

                                for (my $i = 0; $i < 3; $i++){
                                        print "attempt $i for $2 \n";
                                        $result = $p->ping($2);
                                                if ($result){
                                                print "############## 
+$2 is alive at attempt $i\n";
                                                last;
                                                }
                                $p->close();
                                }
                                unless ($result){
                                         print "*****$2 is dead*****\n
+";
                                }
                }

}
[download]

When I run them individually they work ok, but when I try to pipe them doing:
./dhcp_line_dispacher.pl | ./line_process.pl
Nothing happens.
So I test the ./dhcp_line_dispacher.pl and realized than I can`t grab the output.
For example from cli, I do:
./dhcp_line_dispacher.pl | grep ACK And can not get the output in real time.
What should I do to get the output and then process it in other script?
regards,
Leo.

Comment on pipe issue (linux) Select or Download Code

Replies are listed 'Best First'.
Re: pipe issue (linux) by NetWallah (Canon) on May 05, 2016 at 20:33 UTC
If your /var/log/messages is infrequently updated, it may appear that your "dispatcher" script does not produce output. Try adding "tail => 5" to the File::Tail constructor - this will make it behave more like the linux "tail" command, and immediately produce 5 lines of output (default is 0). Try moving the Ping constructor outside the loop. Why not combine the scripts, since they seem inter-dependent. On my system, the DHCPACK line looks like this : `May 5 11:19:59 MyHostName dhcpd: DHCPACK on 192.168.0.94 to 00:50:56: +ab:42:51 (localhost) via eth1` [download] so, your regex-scanner is very dependent on the syntax produced. Double check that it matches the lines generated by your DHCP server. This is not an optical illusion, it just looks like one.	[reply] [d/l]
Re: pipe issue (linux) by haukex (Archbishop) on May 05, 2016 at 20:11 UTC
Hi leostereo, What happens if you pipe the first script to another process (like grep) and then wait a while? If you get output in a large chunk after a wait, you're probably Suffering from Buffering. Add a `$\|=1;` at the top of `dhcp_line_dispacher.pl` to see if it helps. Hope this helps, -- Hauke D	[reply] [d/l] [select]
Re: pipe issue (linux) (buffering) by tye (Sage) on May 06, 2016 at 20:00 UTC
I bet that the biggest contributor to this is that you are "suffering from buffering". When the output from dispatcher is going to a terminal, then that output is "line buffered" which means each line of output appears immediately. When that output is going through a pipe, then it is "fully buffered" which means no output will flow into the pipe until the internal I/O buffer fills up (for example, no output until at least 4KB of output is produced and then the first 4KB of output goes into the pipe at once). The easiest improvement is to just add `$\| = 1;` to your dispatcher script. - tye	[reply] [d/l]
Re: pipe issue (linux) by james28909 (Deacon) on May 05, 2016 at 20:09 UTC
I wonder if something like `my @output = exec(script.pl);` would work? Dont have linux currently installed so cant test anything on the linux side.	[reply] [d/l]
Re^2: pipe issue (linux) by haukex (Archbishop) on May 05, 2016 at 20:23 UTC
Hi james28909, From exec: The exec function executes a system command and never returns You're probably thinking of backticks aka qx, although that will also wait for the completion of the command before returning, so I don't think it'll help here. (BTW, a good alternative to qx is Capture::Tiny.) What might work is opening a pipe (Using open() for IPC) or using a module like IPC::Run (I haven't used the latter myself yet but it seems to be a good module). However, using those in the OP's `line_process.pl` won't fix the problem if `dhcp_line_dispacher.pl` is still buffering its output ;-) Regards, -- Hauke D	[reply] [d/l] [select]