Extract JSON data

omegaweaponZ has asked for the wisdom of the Perl Monks concerning the following question:

I'm looking for an effective way to extract data from a JSON feed in a nice perl parser to dump values into variables for use. Feeds are separated line by line instead of an over-encompassing array, so it's throwing me off. Here is an example:

{
"1": {"subject1": "value", "subject2": [{"subject3": "value", "subject
+4": "value"}], "subject5": "value", "subject6": value, "subject7": "v
+alue"}, 
"2": {"subject1": "value", "subject2": [{"subject3": "value", "subject
+4": "value"}], "subject5": "value", "subject6": value, "subject7": "v
+alue"}, 
"3": {"subject1": "value", "subject2": [{"subject3": "value", "subject
+4": "value"}], "subject5": "value", "subject6": value, "subject7": "v
+alue"},
[download]

etc.... (there is a lot more, ignore malformed JSON!) So instead of something that has an all encompassing array reference such as

DATA [
"1": {"subject1": "value"}, 
"2": {"subject1": "value"}
]
[download]

Which would be easy where I can identify each subject's value by a format similar to the one below, it is just separated essentially by unique numbers line by line from {} which are not detecting the array. My older code had said something along the lines of:

    my $jsonoutput = "Above raw JSON format...";
    my $json = new JSON;
    my $jsontext = $json->allow_nonref->utf8->relaxed->escape_slash->l
+oose->allow_singlequote->allow_barekey->decode("$jsonoutput");
 
 foreach my $stuff(@{$json->{1}}){
      my %hash = ();
      $hash{subject1} = $stuff->{subject1};
}
[download]

Doing this produces "Not an Array Reference". If I was to use "subject2" as oppsed to 1 from $json-> it does not error out, yet does not show subject 3, subject 4, etc's value.

Comment on Extract JSON data Select or Download Code

Replies are listed 'Best First'.
Re: Extract JSON data by roboticus (Chancellor) on May 18, 2013 at 21:07 UTC
omegaweaponZ: I'd suggest using JSON to parse out JSON items. Be aware, however, that the data you show in your example is not valid JSON. Be sure to arrange with the provider of the so-called JSON to have them correct it. The bits I see wrong are: Mismatched curly braces (missing right curly braces) Missing quotes on the value for "subject6" Superfluous commas I don't know whether the commas are a problem or not, but the missing curly braces and missing quotes are problematic. Once you fix the data, the JSON package makes parsing trivial: $ cat t.pl use strict; use warnings; use Data::Dumper; use JSON; my $text; { local $/;$text= <DATA> }; my $junk = decode_json($text); print Dumper($junk); __DATA__ { "1": { "subject1": "value", "subject2": [ { "subject3": "value", "su +bject4": "value" } ], "subject5": "value", "subject6": "value", "subj +ect7": "value" } } $ perl t.pl $VAR1 = { '1' => { 'subject7' => 'value', 'subject2' => [ { 'subject4' => 'value', 'subject3' => 'value' } ], 'subject5' => 'value', 'subject1' => 'value', 'subject6' => 'value' } }; [download] ...roboticus When your only tool is a hammer, all problems look like your thumb.	[reply] [d/l]
Re^2: Extract JSON data by Random_Walk (Prior) on May 19, 2013 at 11:00 UTC
I don't know whether the commas are a problem or not, but the missing curly braces and missing quotes are problematic. Once you fix the data, the JSON package makes parsing trivial: The trailing commas are out of spec, but if you use the `$json->relaxed([$enable])` then they will parse with no problems. `relaxed` also allows # comments in your JSON. Nice for config files. Cheers, R. Pereant, qui ante nos nostra dixerunt!	[reply] [d/l] [select]
Re^2: Extract JSON data by omegaweaponZ (Beadle) on May 21, 2013 at 13:59 UTC
I've updated the above a bit, I might not have explained it properly. Ignore the malformed brackets, this is just a sample code. The issue is when I place in the numerals of 1, 2, 3, etc, any underlying subject comes up as "Not an array reference". If I actually put in subject3, within its own array, that does not error out, but it also does not show me the values of what's within the array (subject 4, 5, etc). So the issue is I need to identify each subjects value, is my array incorrect or am I setting up the parameters incorrectly? Also, the JSON output cannot be changed	[reply]
Re^3: Extract JSON data by omegaweaponZ (Beadle) on May 22, 2013 at 20:28 UTC
I resolved this myself, please ignore. At this point, still need to store the numerals of 1,2,3, however into a variable from reading the json file	[reply]
Re: Extract JSON data by LanX (Saint) on May 18, 2013 at 21:01 UTC
Your JSON is a string, put brackets around them `$new_JSON="[$string]"` before parsing and you have the array structure you were looking for ! =) update works for me, but your "sample data" was malformed! Cheers Rolf ( addicted to the Perl Programming Language)	[reply] [d/l]
Re: Extract JSON data by hdb (Monsignor) on May 18, 2013 at 21:03 UTC
What I find strange is that there is no closing brace corresponding to the opening brace at the beginning of each line.	[reply]

update