Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: Parsing from another program (stream)

by Marshall (Canon)
on Feb 28, 2022 at 23:46 UTC ( [id://11141720]=note: print w/replies, xml ) Need Help??


in reply to Parsing from another program (stream)

First this is a cool application! I went to https://github.com/bemasher/rtlamr to get an idea of what you are doing. I didn't read everything, I just perused enough to get the general idea. I assume that you have started rtl_tcp and that it is running in the background somewhere. I further assume that when you start rtlamr in another command window that you get an output as shown at that URL.

The syntax, "open $fh, '-|', $programName" should work as long as you have the path and file permissions correct (make sure execute permission is enabled). Make sure that rtlamr runs in a command window like the example at the URL.

Replace this code....

my @array; my $i=0; my $Radio_string =""; # Need to launch RTLAMR and read the strings coming out. # Can pre-filter to just MY meter on the commandline with: # cd ~/go/bin # ./rtlamr -msgtype=scm+ --format=json -filterid=101419821, 101419999 +-unique=false # Launch the radio system("/home/jj/go/bin/rtlamr -msgtype=scm+ --format=json -filterid=1 +01419821,101419999 -unique=false"); # slurp the headers do { $Radio_string = $_; Print $i; Print $Radio_string; $i++; } until $i == 10; Print("\n-------End of headers.--------\n\n"); # now wait for and process usage reports #@array = json(system("/home/jj/go/bin/rtlamr -msgtype=scm+ --format=j +son -filterid=101419821,101419999 -unique=false")); $Radio_string = $_; Print $Radio_string; @array = json($Radio_string); Dump(@array);
With this:
use IO::Handle; #for flush method my $rtlamr = "/home/jj/go/bin/rtlamr -msgtype=scm+ --format=json -filt +erid=1 +01419821,101419999 -unique=false"; open my $fh, '-|', $rtlamr or die "unable to start rtlamr program $!"; my $disk_file = "somepath"; open my $disk_fh, '>>', $disk_file or die "unable to open disk history + file! $!"; while (<$fh>) # this is a "blocking" read of radio receiver { print $disk_fh $_; $disk_fh-> flush; # update disk file immediately } __END__ That's it. The analysis program of the disk file, "somepath" is a separate program.
As you have correctly surmised, you will not get an EOF while reading the file handle associated with the radio receiver. Although I believe that killing the rtlamr process would cause an EOF, I don't recommend that you do that.

The loop I show above is infinite - it will never end because there is no EOF. Most of the clock time will be spend just waiting at the while statement with essentially nothing happening otherwise (no CPU cycles for this program). It is possible to do "non-blocking" reads and come up with a scheme to decide that "all the data has been read that can be read at this time". However, that is messy and I don't recommend that.

You can just leave the above program and rl_tcp running all the time. To do the analysis, you should have a 3rd program.

In the analysis program, open the disk file that is being appended by the program above. You can read that filehandle, line by line and you will get an EOF when all of the data has been read. There is nothing wrong with reading a file that is being appended by another program.

However, be aware that very rarely there is the possibility of a premature EOF - meaning that EOF will happen before a newline is encountered. You should allow for that. You can either throw this last line away or just restart the reading process by re-opening the file and reading the file from the beginning again (you just barely missed the last part of the last line).

So, by writing to an intermediate disk file, we solve the "there is no EOF" problem. The disk file will have an EOF when all the data that is currently in it has been read.
Also, rather than counting some fixed number of non-Jason lines to throw away, consider a simple filter, a valid line being one that starts with { and ends with \n. If the last line is a partial one, it won't end in \n.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11141720]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (5)
As of 2024-04-24 07:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found