in reply to polishing up a json fetching script for weather data

WRT Q2, it's not perl but jq is really handy for pretty printing and manipulating JSON data from the command line. Basically you can think of it like awk but for files where the lines are JSON data rather than unstructured text. You can use curl to pull down a sample file, then interactively pick it apart with jq to figure out what bits you're interested in, and then automate whatever in perl.

Additionally: For the warnings, the best way to get rid of them is to not trigger them. Check that you're actually getting values where you expect them before using things (alternately you could still be careless but mute them with no warnings 'undefined' in the smallest possible scope). That being said, they should be going to STDERR so if you use whatever your shell's syntax for that (e.g. myscript blah blah 2>errors).

The cake is a lie.
The cake is a lie.
The cake is a lie.

Replies are listed 'Best First'.
Re^2: polishing up a json fetching script for weather data
by Aldebaran (Curate) on May 21, 2020 at 06:43 UTC
    WRT Q2, it's not perl but jq is really handy for pretty printing and manipulating JSON data from the command line. Basically you can think of it like awk but for files where the lines are JSON data rather than unstructured text. You can use curl to pull down a sample file, then interactively pick it apart with jq to figure out what bits you're interested in, and then automate whatever in perl.

    Thanks for this tip, Fletch, it has really helped me pick apart these things. I folded them into system calls in a script where I built on what worked before:

    #!/usr/bin/perl 

use strict;
use warnings;
use open ':std', OUT => ':utf8';

#my $file = '1.weather.json';
#system ("cat $file | jq '[.properties]' | more");
#system ("cat $file | jq '[.properties.cloudLayers]' >2.txt");
#system ("cat $file | jq '[.properties.temperature]' >3.txt");
# these were effective commands

##different json, same host
my $file = 'json_stuff/1.openapi.json';
#system ("cat $file | jq '[.]' >5.txt");
#system ("cat $file | jq '[.paths]' >6.txt");
system ("cat $file | jq '[.paths.get]' >7.txt");
                                      
__END__

    [download]
    Additionally: For the warnings, the best way to get rid of them is to not trigger them. Check that you're actually getting values where you expect them before using things (alternately you could still be careless but mute them with no warnings 'undefined' in the smallest possible scope). That being said, they should be going to STDERR so if you use whatever your shell's syntax for that (e.g. myscript blah blah 2>errors).

    That's the ticket.

    $ ./6.weather.pl 2>errors
[reply]
[d/l]
[select]
      system ("cat $file | jq '[.paths.get]' >7.txt");

      I very strongly recommend against doing it this way, for several reasons: First, you're using the shell's features, namely piping a file into jq when that's not necessary - jq filter files works just as well. Second, you're using the single-argument form of system, and interpolating a variable into that, opening yourself up to all sorts of possible issues; I describe that and several better alternatives here, in this case I might suggest capturex from IPC::System::Simple. Third, calling an external command in the first place - everything that jq can do, you can do just as well in Perl, by parsing the JSON file into a Perl data structure and then working with that data structure.

[reply]
[d/l]
[select]
        I very strongly recommend against doing it this way, for several reasons: First, you're using the shell's features, namely piping a file into jq when that's not necessary - jq filter files works just as well.

        I've read and worked through your post on varying system calls and am glad to have another link to it. The system call was a band-aid, because jq can't seem to do anything directly:

        $ pwd
/home/hogan/Documents/hogan/json_stuff
$ ls
1.openapi.json  2.weather.json      russian-words.json
1.weather.json  countries.geo.json  snapshots
$ jq '.' 1.openapi.json 
jq: error: Could not open file 1.openapi.json: Permission denied
$

        [download]

        Apparently I used 'snap' to install it:

        $ history | grep jq
 1376  curl 'https://api.github.com/repos/stedolan/jq/commits?per_page
+=5'
 1377  jq '.[] | {message: .commit.message, name: .commit.committer.na
+me}'
 1378  sudo snap install jq

        [download]

        I tried to raise priveleges with:

         1429  chmod a+x snap/bin/jq
 1430  chmod a+x /snap/bin/jq
 1431  sudo chmod a+x /snap/bin/jq

        [download]

        , without success (??)

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom