Re^2: polishing up a json fetching script for weather data

WRT Q2, it's not perl but jq is really handy for pretty printing and manipulating JSON data from the command line. Basically you can think of it like awk but for files where the lines are JSON data rather than unstructured text. You can use curl to pull down a sample file, then interactively pick it apart with jq to figure out what bits you're interested in, and then automate whatever in perl.

Thanks for this tip, Fletch, it has really helped me pick apart these things. I folded them into system calls in a script where I built on what worked before:

#!/usr/bin/perl 

use strict;
use warnings;
use open ':std', OUT => ':utf8';

#my $file = '1.weather.json';
#system ("cat $file | jq '[.properties]' | more");
#system ("cat $file | jq '[.properties.cloudLayers]' >2.txt");
#system ("cat $file | jq '[.properties.temperature]' >3.txt");
# these were effective commands

##different json, same host
my $file = 'json_stuff/1.openapi.json';
#system ("cat $file | jq '[.]' >5.txt");
#system ("cat $file | jq '[.paths]' >6.txt");
system ("cat $file | jq '[.paths.get]' >7.txt");
                                      
__END__
[download]

Additionally: For the warnings, the best way to get rid of them is to not trigger them. Check that you're actually getting values where you expect them before using things (alternately you could still be careless but mute them with no warnings 'undefined' in the smallest possible scope). That being said, they should be going to STDERR so if you use whatever your shell's syntax for that (e.g. myscript blah blah 2>errors).

That's the ticket.

$ ./6.weather.pl 2>errors

Comment on Re^2: polishing up a json fetching script for weather data Select or Download Code

Replies are listed 'Best First'.
Re^3: polishing up a json fetching script for weather data by haukex (Archbishop) on May 22, 2020 at 08:18 UTC
`system ("cat $file \| jq '[.paths.get]' >7.txt");` I very strongly recommend against doing it this way, for several reasons: First, you're using the shell's features, namely piping a file into `jq` when that's not necessary - `jq filter files` works just as well. Second, you're using the single-argument form of system, and interpolating a variable into that, opening yourself up to all sorts of possible issues; I describe that and several better alternatives here, in this case I might suggest `capturex` from IPC::System::Simple. Third, calling an external command in the first place - everything that `jq` can do, you can do just as well in Perl, by parsing the JSON file into a Perl data structure and then working with that data structure.	[reply] [d/l] [select]
Re^4: polishing up a json fetching script for weather data by Aldebaran (Curate) on May 23, 2020 at 08:20 UTC
I very strongly recommend against doing it this way, for several reasons: First, you're using the shell's features, namely piping a file into jq when that's not necessary - jq filter files works just as well. I've read and worked through your post on varying system calls and am glad to have another link to it. The system call was a band-aid, because jq can't seem to do anything directly: `$ pwd /home/hogan/Documents/hogan/json_stuff $ ls 1.openapi.json 2.weather.json russian-words.json 1.weather.json countries.geo.json snapshots $ jq '.' 1.openapi.json jq: error: Could not open file 1.openapi.json: Permission denied $` [download] Apparently I used 'snap' to install it: `$ history \| grep jq 1376 curl 'https://api.github.com/repos/stedolan/jq/commits?per_page +=5' 1377 jq '.[] \| {message: .commit.message, name: .commit.committer.na +me}' 1378 sudo snap install jq` [download] I tried to raise priveleges with: `1429 chmod a+x snap/bin/jq 1430 chmod a+x /snap/bin/jq 1431 sudo chmod a+x /snap/bin/jq` [download] , without success (??)	[reply] [d/l] [select]
Re^5: polishing up a json fetching script for weather data by haukex (Archbishop) on May 23, 2020 at 08:23 UTC
`jq: error: Could not open file 1.openapi.json: Permission denied` It's complaining about the file `1.openapi.json`, not the `jq` executable; try `chmod a+r 1.openapi.json` (if you don't mind giving everyone read access).	[reply] [d/l] [select]