anautismobserver has asked for the wisdom of the Perl Monks concerning the following question:

EDITED to add: this has been solved. Thanks so much.

I'm a perl novice who wants to learn just enough perl to get by without putting much effort into it.

Please tell me how to read the JSON object returned by

http://web.archive.org/cdx/search/cdx?url=archive.org&output=json&limi +t=3
obtained from
https://github.com/internetarchive/wayback/tree/master/wayback-cdx-ser +ver#output-format-json

CLARIFICATION ADDED: Below is an example json object. I want to assign a variable $first_timestamp to the first timestamp (19970126045828).

[["urlkey","timestamp","original","mimetype","statuscode","digest","le +ngth"], ["org,archive)/", "19970126045828", "http://www.archive.org:80/", "te +xt/html", "200", "Q4YULN754FHV2U6Q5JUT6Q2P57WEWNNY", "1415"], ["org,archive)/", "19971011050034", "http://www.archive.org:80/", "te +xt/html", "200", "XAHDNHZ5P3GSSSNJ3DMEOJF7BMCCPZR3", "1402"], ["org,archive)/", "19971211122953", "http://www.archive.org:80/", "te +xt/html", "200", "XAHDNHZ5P3GSSSNJ3DMEOJF7BMCCPZR3", "1405"]]

How do I do that?

Context:

I currently have the following working code:

use strict; use warnings; use Mojo::UserAgent; use LWP::UserAgent; use HTTP::Request::Common qw(GET); my $outputFilename = "test_output.txt"; my $fh_data; my $name; open( $fh_data, '>', $outputFilename ) or die "Could not open file '$outputFilename' $!"; my $url = "https://public-api.wordpress.com/rest/v1/read/feed/?url=anautismobser +ver.wordpress.com"; my $ua = Mojo::UserAgent->new; my $json = $ua->get($url)->res->json; for my $feedurl ( @{ $json->{feeds} } ) { if ( $feedurl->{meta}{links}{site} ) { my $siteurl2 = $feedurl->{meta}{links}{site}; my $ua2 = Mojo::UserAgent->new; my $json3 = $ua2->get($siteurl2)->res->json; $name = $json3->{name}; print $fh_data "$name"; } } close $fh_data;

I want to modify the code to read the JSON object returned if I set

my $url = http://web.archive.org/cdx/search/cdx?url=archive.org&output +=json&limit=3

and I want to write the timestamps into a text file.

Thank you.

Replies are listed 'Best First'.
Re: reading a JSON object
by Corion (Patriarch) on May 08, 2022 at 19:22 UTC

    I'm not sure what your question is. You are already reading the JSON data and extracting parts from it:

    my $json = $ua->get($url)->res->json; ... my $json3 = $ua2->get($siteurl2)->res->json;

    Maybe you can tell us where exactly you have problems, and show the relevant input data and what you expect your program to do?

      I edited my original question to add a clarification.

        The new URL provides a different response, the code posted looks like it's based upon the code from a previous question some time ago. You need to alter the code to print the data. Which part are you experiencing problems with? Are you perhaps confused by how to address the fields in the JSON response?

      I want to extract the timestamps and write them into a text file.
Re: reading a JSON object
by AnomalousMonk (Archbishop) on May 09, 2022 at 04:51 UTC

    Re: Clarification:

    Win8 Strawberry 5.30.3.1 (64) Mon 05/09/2022 0:45:49 C:\@Work\Perl\monks >perl use strict; use warnings; use JSON; use Data::Dump qw(dd); my $json_string = <<'EOJ'; [["urlkey","timestamp","original","mimetype","statuscode","digest","le +ngth"], ["org,archive)/", "19970126045828", "http://www.archive.org:80/", "text/html", "200", "Q4YULN754FHV2U6Q5JUT6Q2P57WEWNNY", "1415"], ["org,archive)/", "19971011050034", "http://www.archive.org:80/", "text/html", "200", "XAHDNHZ5P3GSSSNJ3DMEOJF7BMCCPZR3", "1402"], ["org,archive)/", "19971211122953", "http://www.archive.org:80/", "text/html", "200", "XAHDNHZ5P3GSSSNJ3DMEOJF7BMCCPZR3", "1405"]] EOJ my $ar_decoded_json = decode_json $json_string; # dd $ar_decoded_json; # for debug my $first_timestamp = $ar_decoded_json->[1][1]; print "'$first_timestamp' \n"; ^Z '19970126045828'


    Give a man a fish:  <%-{-{-{-<

      Thanks. That's exactly the information I needed.

      EDITED to add: the question below has been answered. Thanks so much.

      Now I have another ignorant question. Below is my current working code. How do I get the size of the JSON object $json_string, and how do I loop over the first index?

      use strict; use warnings; use Mojo::UserAgent; use LWP::UserAgent; use HTTP::Request::Common qw(GET); # use JSON; use Data::Dump qw(dd); my $outputFilename = "test_output.txt"; my $fh_data; my $urlkey_result; my $timestamp_result; my $original_result; my $mimetype_result; my $statuscode_result; my $digest_result; my $length_result; open( $fh_data, '>', $outputFilename ) or die "Could not open file '$outputFilename' $!"; my $url = "http://web.archive.org/cdx/search/cdx?url=anautismobserver.wordpress. +com&output=json"; # &limit=3 my $ua = Mojo::UserAgent->new; my $json_string = $ua->get($url)->res->json; # dd $json_string; # for debug $urlkey_result = $json_string->[0][0]; $timestamp_result = $json_string->[0][1]; $original_result = $json_string->[0][2]; $mimetype_result = $json_string->[0][3]; $statuscode_result = $json_string->[0][4]; $digest_result = $json_string->[0][5]; $length_result = $json_string->[0][6]; print $fh_data "\n$urlkey_result\t$timestamp_result\t$original_result\ +t$mimetype_result\t$statuscode_result\t$digest_result\t$length_result +"; $urlkey_result = $json_string->[1][0]; $timestamp_result = $json_string->[1][1]; $original_result = $json_string->[1][2]; $mimetype_result = $json_string->[1][3]; $statuscode_result = $json_string->[1][4]; $digest_result = $json_string->[1][5]; $length_result = $json_string->[1][6]; print $fh_data "\n$urlkey_result\t$timestamp_result\t$original_result\ +t$mimetype_result\t$statuscode_result\t$digest_result\t$length_result +"; $urlkey_result = $json_string->[2][0]; $timestamp_result = $json_string->[2][1]; $original_result = $json_string->[2][2]; $mimetype_result = $json_string->[2][3]; $statuscode_result = $json_string->[2][4]; $digest_result = $json_string->[2][5]; $length_result = $json_string->[2][6]; print $fh_data "\n$urlkey_result\t$timestamp_result\t$original_result\ +t$mimetype_result\t$statuscode_result\t$digest_result\t$length_result +"; $urlkey_result = $json_string->[3][0]; $timestamp_result = $json_string->[3][1]; $original_result = $json_string->[3][2]; $mimetype_result = $json_string->[3][3]; $statuscode_result = $json_string->[3][4]; $digest_result = $json_string->[3][5]; $length_result = $json_string->[3][6]; print $fh_data "\n$urlkey_result\t$timestamp_result\t$original_result\ +t$mimetype_result\t$statuscode_result\t$digest_result\t$length_result +"; close $fh_data;

        I'm confused.
            my $json_string = $ua->get($url)->res->json;
        Is $json_string in fact a string, or is it a decoded JSON object as shown in the example here, i.e., a reference to an array of arrays (in that particular example)? Your use of the $json_string->[0][0] syntax suggests the latter.

        If $json_string is in fact a ref. to an AoA, you might do something like this:

        Win8 Strawberry 5.30.3.1 (64) Mon 05/09/2022 14:30:45 C:\@Work\Perl\monks >perl use strict; use warnings; use JSON; use Data::Dump qw(dd); my $json_string = <<'EOJ'; [["urlkey","timestamp","original","mimetype","statuscode","digest","le +ngth"], ["org,archive)/", "19970126045828", "http://www.archive.org:80/", "text/html", "200", "Q4YULN754FHV2U6Q5JUT6Q2P57WEWNNY", "1415"], ["org,archive)/", "19971011050034", "http://www.archive.org:80/", "text/html", "200", "XAHDNHZ5P3GSSSNJ3DMEOJF7BMCCPZR3", "1402"], ["org,archive)/", "19971211122953", "http://www.archive.org:80/", "text/html", "200", "XAHDNHZ5P3GSSSNJ3DMEOJF7BMCCPZR3", "1405"]] EOJ my $ar_decoded_json = decode_json $json_string; # dd $ar_decoded_json; # for debug for my $ar_jentry (@$ar_decoded_json) { # handle each decoded JSON en +try printf "\n'%s' '%s' '%s' '%s' '%s' '%s' '%s'", @$ar_jentry; } ^Z 'urlkey' 'timestamp' 'original' 'mimetype' 'statuscode' 'digest' 'leng +th' 'org,archive)/' '19970126045828' 'http://www.archive.org:80/' 'text/ht +ml' '200' 'Q4YULN754FHV2U6Q5JU T6Q2P57WEWNNY' '1415' 'org,archive)/' '19971011050034' 'http://www.archive.org:80/' 'text/ht +ml' '200' 'XAHDNHZ5P3GSSSNJ3DM EOJF7BMCCPZR3' '1402' 'org,archive)/' '19971211122953' 'http://www.archive.org:80/' 'text/ht +ml' '200' 'XAHDNHZ5P3GSSSNJ3DM EOJF7BMCCPZR3' '1405'
        (I used different formatting just to try to make things fit better and to clearly delineate the strings. Sorry about the wraparound.)


        Give a man a fish:  <%-{-{-{-<

      How do I convert timestamps to numbers so I can subtract them without getting the following error message:

      Argument "timestamp" isn't numeric in subtraction

        The error message indicates you're using the string timestamp in subtraction.

        $ perl -wE 'say "timestamp" - 2' Argument "timestamp" isn't numeric in subtraction (-) at -e line 1. -2

        I don't see how the string can be meaningfully converted to a number. Please, show the code that demonstrates the problem. Are you using the string directly instead of using it as a hash key?

        map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

        Note that the first row in your array consists of the column names. Maybe you are not skipping that first row properly?

        Other than that, choroba's advice of showing the relevant code and data still applies.