I (believe) I am trying to parse from an unterminated string returned from an application.
Enviroment: Ubuntu linux, Eclipse:EPIC as workspace

The application: I am reading my gas meter (by radio) using a Software Defined Radio.
The code for it can be found here: https://github.com/bemasher/rtlamr
Since the output requires the hardware, I'll include a sample below.

The output code looks like this:
10:33:46.213965 decode.go:45: CenterFreq: 912600155
10:33:46.215377 decode.go:46: SampleRate: 2359296
10:33:46.215528 decode.go:47: DataRate: 32768
10:33:46.215651 decode.go:48: ChipLength: 72
10:33:46.215780 decode.go:49: PreambleSymbols: 16
10:33:46.215903 decode.go:50: PreambleLength: 2304
10:33:46.216021 decode.go:51: PacketSymbols: 128
10:33:46.216137 decode.go:52: PacketLength: 18432
10:33:46.216263 decode.go:59: Protocols: scm+
10:33:46.216404 decode.go:60: Preambles: 0001011010100011
10:33:46.216529 main.go:126: GainCount: 29
{"Time":"2022-02-26T10:35:55.66825429-05:00","Offset":0,"Length":0,"Ty
+pe":"SCM+","Message":{"FrameSync":5795,"ProtocolID":30,"EndpointType"
+:188,"EndpointID":101419821,"Consumption":305782,"Tamper":2056,"Packe
+tCRC":26848}}
{"Time":"2022-02-26T10:36:10.667844535-05:00","Offset":0,"Length":0,"T
+ype":"SCM+","Message":{"FrameSync":5795,"ProtocolID":30,"EndpointType
+":188,"EndpointID":101419821,"Consumption":305782,"Tamper":2056,"Pack
+etCRC":26848}}
...

[download]


The first eleven lines are only presented when first run, the remaining JSON strings are the actual meter readings, and the information I want to parse.
The results continue forever, so there is no EOF to signal that it's time to process.
I can slurp it in using this code (If you wonder, rtl_tcp has been started in another window):
<br><br>
#!/usr/bin/perl -w
# Routines to pull data from the Gas meter.
# 2/15 jj v0.0.1
#
use strict;
use warnings;
use JSON;
use Data::Dumper;

use Log::Log4perl qw(:easy);
# Initialize Logger
my $log_conf = q(
   log4perl.rootLogger              = DEBUG, LOG1
   log4perl.appender.LOG1           = Log::Log4perl::Appender::File
   log4perl.appender.LOG1.filename  = /home/jj/Documents/log/Meterlog.
+log
   log4perl.appender.LOG1.mode      = append
   log4perl.appender.LOG1.layout    = Log::Log4perl::Layout::PatternLa
+yout
   log4perl.appender.LOG1.layout.ConversionPattern = %d %p %m %n
);
Log::Log4perl::init(\$log_conf);
my $logger = Log::Log4perl->get_logger();

# sample logging statement
$logger->info("this is an info log message");

my @array;
my $i=0;
my $Radio_string ="";

# Need to launch RTLAMR and read the strings coming out.
#  Can pre-filter to just MY meter on the commandline with:
# cd ~/go/bin
# ./rtlamr -msgtype=scm+ --format=json -filterid=101419821, 101419999 
+-unique=false

# Launch the radio
system("/home/jj/go/bin/rtlamr -msgtype=scm+ --format=json -filterid=1
+01419821,101419999 -unique=false");
# slurp the headers
do {
    $Radio_string = $_;
    Print $i;
    Print $Radio_string;
    $i++;
} until $i == 10;

Print("\n-------End of headers.--------\n\n");

# now wait for and process usage reports
#@array = json(system("/home/jj/go/bin/rtlamr -msgtype=scm+ --format=j
+son -filterid=101419821,101419999 -unique=false"));  
    $Radio_string = $_;
    Print $Radio_string;
    @array = json($Radio_string);
Dump(@array);

[download]


In this code, the code beyond the "# slurp.." never is seen. It may or may not work, I included it to indicate what I was EXPECTING to do.
From this, I would expect that the first couple header lines would be pulled and printed. Then I could get my data.
That's not what I get. It appears as an *unterminated* string coming in "header0\nheader1\nheader2....datastring0\ndatastring\n....." and is printed to the console.

Things I tried:
exec - Doesn't work (by def'n), Perl program waits until command exits.
system - same thing, except a 'fork' is wrapped around exec. Still needs to exit to return a value.
both start a process that I need to 'kill', as rtlamr is orphaned. (I run 'top | grep rtlamr' to find the orphan.)
qx or backtick - same thing.
The archives give a lot of good examples of how to use each these, but in each case, the called program *ends* so the returned data is processed upon the return from the called program.

Tried open:
print "spawned:\n";
chdir ("/home/jj/go/bin/") ;
if ( -e "rtlamr" ) {print "Found rtlamr.\n";}
$pid = open($handle, "rtlamr -msgtype=scm+ --format=json -filterid=101
+419821,101419999 -unique=false" ) or die "Couldn't open file rtlamr, 
+$!";

[download]

and I got:
spawned:<br>
Found rtlamr.
Couldn't open file rtlamr, No such file or directory at /home/jj/eclip
+se-workspace/ReadGas/ReadFromRadio.pl line 49.

[download]


open ( $handle, ">", "rtlamr -msgtype=scm+ --format=json -filterid=101
+419821,101419999 -unique=false") or die "Can't open > output.txt: $!"
+ ;

[download]

Works! (kinda)
So does:
open my $handle, ">", "rtlamr -msgtype=scm+ --format=json -filterid=101419821,101419999 -unique=false" ;
....in that they both 'return' and don't simply hang. But I'm still looking for a return value.
tried open piping the output ie:
open my $handle, "-|", "rtlamr -msgtype=scm+ --format=json -filterid=101419821,101419999 -unique=false" ;
Got this:
spawned:
Found rtlamr.
loop#:0
Can't exec "rtlamr": No such file or directory at /home/jj/eclipse-wor
+kspace/ReadGas/ReadFromRadio.pl line 50.
print() on closed filehandle $handle at /usr/lib/x86_64-linux-gnu/perl
+/5.30/IO/Handle.pm line 431.
loop#:1 ......

[download]

Added the 'or die' and found:
Found rtlamr.
Can't exec "rtlamr": No such file or directory at /home/jj/eclipse-wor
+kspace/ReadGas/ReadFromRadio.pl line 50.
Couldn't open file rtlamr, No such file or directory at /home/jj/eclip
+se-workspace/ReadGas/ReadFromRadio.pl line 50.

[download]


From this, I suspect that 'open' is not really what I want.
I'm not looking at a file, but STDOUT (am I wrong?) from rtlamr.
And it is never really 'opened' and since it is an executable, I don't think 'open' is the right path.

At this point, I think I have exhausted the scrolls.
(Other seekers, check out "Threads, Multi-Processing, and Inter-Process Communication" (https://www.perlmonks.org/?node_id=591896). Good stuff.)
Paths untraveled: IPC::Run, IPC::Open3. Have I missed it?

Please be kind. I have coded, but been on the sidelines for over 20 yrs, and I am rusty.
I have tried to be verbose for the next seeker of knowledge.
Your wisdom is eagerly welcome.

In reply to Parsing from another program (stream) by HiredGunTech

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.