reading from a named pipe

skendric has asked for the wisdom of the Perl Monks concerning the following question:

I use an application (Risto Vaarandi's Simple Event Correlator) to grab strings from /var/log/syslog and write them to a named pipe

From sec.conf

type=Single
ptype=regexp
pattern=T(\d\d:\d\d:\d\d).*?(\S+)\s+nodewatch.*?: (.*)
desc=NodeWatch on $2 at $1
action=write /home/tocops/.tocpipe nodewatch $1 $3
[download]

And then I wrote an application (toclogd) which reads from this pipe and does useful things with what it reads https://www.skendric.com/nmgmt/console/toc/toclogd

Relevant section inserted here

[...]
{
  # Declare variables
  my $fifo;         # The pipe from which we'll read
  my $input;        # Text which SEC handed us
  my $clean;        # $input stripped of annoying characters
  my $text;         # $clean, stripped of the log name

  MAIN_LOOP:
  while (1) {

    # Open pipe
    ualarm(0);
    unless (open $fifo, '<', $pipe) {
      log_it("Cannot open $pipe: $!");
      sleep 5;
      next MAIN_LOOP;
    }

    # Give ourselves .01s to read input
    ualarm(10000);
    $input = <$fifo>;
    next MAIN_LOOP unless defined $input;
    chomp $input;

    # Process input
    alarm(0);
    next MAIN_LOOP unless $clean = strip_junk($input);
    next MAIN_LOOP unless $log   = find_log_name($clean);
    ($text) = ($clean) =~ /^\w+\s+(.*)/;
    write_line($text);
  }
}
[...]
[download]

This has worked great under RHEL derivatives for ... almost two decades now: I currently use it under Rocky 8.9, for example

However, I have started to experiment with Ubuntu (22.04 Jammy Jellyfish), and I see an odd effect

When SEC writes a couple messages in rapid success to .tocpipe, toclogd only picks up the first one

In other words, under Rocky 8.9, toclogd reads both of these messages and does something useful with both of them

2024-06-01T05:43:27.241457-07:00 rocky sec[123632]: Writing event 'nod
+ewatch 05:43:26 flem [+3]' to file '/home/tocops/.tocpipe'
2024-06-01T05:43:27.241858-07:00 rocky sec[123632]: Writing event 'nod
+ewatch 05:43:26 flem [up]' to file '/home/tocops/.tocpipe'
[download]

However, under Ubuntu, toclogd only does something useful with the first of these; it appears to miss the second

2024-06-01T05:43:26.119268-07:00 ubuntu sec[664336]: Writing event 'no
+dewatch 05:43:26 flem [+3]' to file '/home/tocops/.tocpipe'
2024-06-01T05:43:26.119887-07:00 ubuntu sec[664336]: Writing event 'no
+dewatch 05:43:26 flem [up]' to file '/home/tocops/.tocpipe'
[download]

Does this a ring a bell for anyone? What might Ubuntu be doing differently than Rhel, wrt named pipes?

Rocky 8.9:     perl-5.34.2
Ubuntu 22.04:  perl-5.34.0
[download]

--sk

Comment on reading from a named pipe Select or Download Code

Replies are listed 'Best First'.
Re: reading from a named pipe by tonyc (Hermit) on Jun 02, 2024 at 23:11 UTC
I don't know why it works on one but not the other, but I can see a possible problem if two messages arrive in quick succession You read only a single line from the pipe on each open, but you're using buffered input, so it's possible if two messages arrive in quick succession that for the `<$fifo>` PerlIO reads both lines into the buffer, and readline() (aka `<>`) returns the first of those two lines You then re-open the pipe each time around the loop and discard the buffered line. Given what you've shown I don't see why you don't just have a classic `while (my $line = <$fifo>) { ... }` loop.	[reply] [d/l] [select]
Re^2: reading from a named pipe by skendric (Novice) on Jun 10, 2024 at 12:46 UTC
Dang, I was getting too complicated there, with the alarms and opening and closing the pipe ... simplifying per your suggestion has fixed the issue under Ubuntu, and it still works fine under Rocky `# Open pipe OPENPIPE: unless (open $fifo, '<', $pipe) { log_it("Cannot open $pipe: $!"); sleep 5; next OPENPIPE; } READPIPE: while (my $line = <$fifo>) { next READPIPE unless defined $line; chomp $line; # Process line next READPIPE unless $clean = strip_junk($line); next READPIPE unless $log = find_log_name($clean); ($text) = ($clean) =~ /^\w+\s+(.*)/; write_line($text); }` [download] Thank you, --sk	[reply] [d/l]