eny has asked for the wisdom of the Perl Monks concerning the following question:

Hello all,

I receive such json file as input. As you can see it is an array of objects where each object is an array of sub-objects. Brief it is a collection of nested objects and arrays. I don't know in advance the structure, the name of the keys and the levels of nesting. Or even it can also be the other way around, a collection of objects with arrays nested on it. The only thing I should know is the object identifier "key name" found at level 2 (in my example "Obj11Id", "Obj12Id", etc). My goal is for each pair of key name and value, to have the "full path" of the key name and the key value. By "full path" I understand the name of the above keys in the structure.

Example: for the pair "Obj11AttributesObj1Key1": "1", the "full path" would be: "Obj1.Obj11Attributes.Obj11AttributesObj1.Obj11AttributesObj1Key1" (dot "." being the separator of the above keys).

I'm new in Perl, I've tried many ways to obtain this info but failed. I'm using the library JSON::PP and function decode_json.

Thank you

The input json file:

[ { "Obj1": [ { "Obj11Id": "Id11", "Obj11Version": "v1", "Obj11Attributes": { "Obj11AttributesObj1": { "Obj11AttributesObj1Key1": "1", "Obj11AttributesObj1Key2": "3" }, "Obj11AttributesObj2": { "Obj11AttributesObj2Key1": "9" } } }, { "Obj12Id": "Id12", "Obj12Version": "v1", "Obj12Attributes": { "Obj12AttributesObj1": { "Obj12AttributesObj1Key1": "2", "Obj12AttributesObj1Key2": "4" }, "Obj12AttributesObj2": { "Obj12AttributesObj2Key1": "8" }, "Obj12AttributesArray1": [ { "Obj12AttributesArray1Obj1Key1": 1 } ] } } ] }, { "Obj2": [ { "Obj21Id": "Id21", "Obj21Version": "v1", "Obj21Attributes": { "Obj21AttributesObj1": { "Obj21AttributesObj1Key1": "3", "Obj21AttributesObj1Key2": "5" }, "Obj21AttributesObj2": { "Obj21AttributesObj2Key1": "7" } } }, { "Obj22Id": "Id22", "Obj22Version": "v1", "Obj22Attributes": { "Obj22AttributesObj1": { "Obj22AttributesObj1Key1": "4", "Obj22AttributesObj1Key2": "6" }, "Obj22AttributesObj2": { "Obj22AttributesObj2Key1": "0" }, "Obj22AttributesArray1": [ { "Obj22AttributesArray1Obj1Key1": 1 } ] } } ] } ]

Replies are listed 'Best First'.
Re: Obtain the full path of a json key
by haukex (Archbishop) on Jun 22, 2025 at 07:46 UTC
    I've tried many ways to obtain this info but failed. I'm using the library JSON::PP and function decode_json.

    It would be best if you showed us what you tried and how it failed (SSCCE), because otherwise we can't tell if you're just asking us do your homework for you or (ab)using us as a code writing service. But since this was a nice little excercise, here's a freebie...

    use warnings; use strict; use JSON::PP qw/decode_json/; # search a data structure recursively for a specific hash key # and return its "path" as a string if it is found sub get_path_by_key { my ($data, $search_key) = @_; # an array of the data items left to search, each being a hash # with keys "d" for the data item and "p" for the item's path: my @to_search = ( { d=>$data, p=>[] } ); while (@to_search) { # get the item we're currently inspecting: my $cur = shift @to_search; if ( ref $$cur{d} eq 'ARRAY' ) { # if the data item is an array, go through its indices for my $i ( 0 .. $#{$$cur{d}} ) { # skip any array values that aren't nested data: next unless ref $cur->{d}[$i]; # put each of the array's values on the search list push @to_search, { d=>$cur->{d}[$i], # this means the output won't include indices: p=>$$cur{p} }; # this would be one way to include indices: #p=>[ @{$$cur{p}}, "[$i]" ] }; } } elsif ( ref $$cur{d} eq 'HASH' ) { # if the current data item is a hash, # first check if it contains the search key: if ( exists $cur->{d}{$search_key} ) { # this builds the return value string: return join '.', @{$$cur{p}}, $search_key; } # otherwise, go through the hash's keys: for my $k ( sort keys %{$$cur{d}} ) { # skip any hash values that aren't nested data: next unless ref $cur->{d}{$k}; # put each of the hash's values on the search list push @to_search, { d=>$cur->{d}{$k}, p=>[ @{$$cur{p}}, $k ] }; } } } return # return nothing in case the key isn't found } # this is just an idiom to read the whole __DATA__ section: my $json = do { local $/; <DATA> }; # decode the JSON string into a Perl data structure: my $data = decode_json($json); # use our function: my $path = get_path_by_key($data, 'Obj11AttributesObj1Key1'); print "path=$path\n"; __DATA__ [ { "Obj1": [ { "Obj11Id": "Id11", "Obj11Version": "v1", "Obj11Attributes": { "Obj11AttributesObj1": { "Obj11AttributesObj1Key1": "1", "Obj11AttributesObj1Key2": "3" }, "Obj11AttributesObj2": { "Obj11AttributesObj2Key1": "9" } } }, { "Obj12Id": "Id12", "Obj12Version": "v1", "Obj12Attributes": { "Obj12AttributesObj1": { "Obj12AttributesObj1Key1": "2", "Obj12AttributesObj1Key2": "4" }, "Obj12AttributesObj2": { "Obj12AttributesObj2Key1": "8" }, "Obj12AttributesArray1": [ { "Obj12AttributesArray1Obj1Key1": 1 } ] } } ] }, { "Obj2": [ { "Obj21Id": "Id21", "Obj21Version": "v1", "Obj21Attributes": { "Obj21AttributesObj1": { "Obj21AttributesObj1Key1": "3", "Obj21AttributesObj1Key2": "5" }, "Obj21AttributesObj2": { "Obj21AttributesObj2Key1": "7" } } }, { "Obj22Id": "Id22", "Obj22Version": "v1", "Obj22Attributes": { "Obj22AttributesObj1": { "Obj22AttributesObj1Key1": "4", "Obj22AttributesObj1Key2": "6" }, "Obj22AttributesObj2": { "Obj22AttributesObj2Key1": "0" }, "Obj22AttributesArray1": [ { "Obj22AttributesArray1Obj1Key1": 1 } ] } } ] } ]

    If you're new to Perl you'll probably need perlreftut to understand much of the code.

Re: Obtain the full path of a json key (gron)
by Arunbear (Prior) on Jun 22, 2025 at 10:56 UTC
    There is a tool called gron which makes JSON greppable. So with your example,
    % gron 11165436.json | head json = []; json[0] = {}; json[0].Obj1 = []; json[0].Obj1[0] = {}; json[0].Obj1[0].Obj11Attributes = {}; json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj1 = {}; json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj1.Obj11AttributesObj +1Key1 = "1"; json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj1.Obj11AttributesObj +1Key2 = "3"; json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj2 = {}; json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj2.Obj11AttributesObj +2Key1 = "9";
    Then, if you just wanted a list of keys, you could strip away the values,
    % gron 11165436.json | perl -pE 's/ =.+//' | head json json[0] json[0].Obj1 json[0].Obj1[0] json[0].Obj1[0].Obj11Attributes json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj1 json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj1.Obj11AttributesObj +1Key1 json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj1.Obj11AttributesObj +1Key2 json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj2 json[0].Obj1[0].Obj11Attributes.Obj11AttributesObj2.Obj11AttributesObj +2Key1
Re: Obtain the full path of a json key
by Corion (Patriarch) on Jun 22, 2025 at 08:26 UTC

    The easiest way is to enumerate all paths and then only return those that match your criteria:

    #!perl use 5.020; use feature 'signatures', 'postderef'; no warnings 'experimental::signatures'; sub all_json_paths( $obj, $prefix = '' ) { #say "$prefix|$obj"; if( !ref $obj ) { return $prefix } elsif( ref $obj eq 'ARRAY' ) { my @res; my $idx = 1; # 1-based or 0-based indexing? for my $el ($obj->@*) { push @res, all_json_paths( $el, $prefix . "[$idx]" ); $idx++; } return @res } elsif( ref $obj eq 'HASH' ) { my @res; for my $k (sort keys $obj->%*) { push @res, all_json_paths( $obj->{$k}, $prefix . ".$k" ); } return @res } } sub path_to_key( $key, $obj ) { my @p = all_json_paths( $obj ); return grep /\Q$key\E$/, all_json_paths( $obj ) } my $struct = [ { "Obj1"=> [ { "Obj11Id"=> "Id11", "Obj11Version"=> "v1", "Obj11Attributes"=> { "Obj11AttributesObj1"=> { "Obj11AttributesObj1Key1"=> "1", "Obj11AttributesObj1Key2"=> "3" }, "Obj11AttributesObj2"=> { "Obj11AttributesObj2Key1"=> "9" } } }, { "Obj12Id"=> "Id12", "Obj12Version"=> "v1", "Obj12Attributes"=> { "Obj12AttributesObj1"=> { "Obj12AttributesObj1Key1"=> "2", "Obj12AttributesObj1Key2"=> "4" }, "Obj12AttributesObj2"=> { "Obj12AttributesObj2Key1"=> "8" }, "Obj12AttributesArray1"=> [ { "Obj12AttributesArray1Obj1Key1"=> 1 } ] } } ] }, { "Obj2"=> [ { "Obj21Id"=> "Id21", "Obj21Version"=> "v1", "Obj21Attributes"=> { "Obj21AttributesObj1"=> { "Obj21AttributesObj1Key1"=> "3", "Obj21AttributesObj1Key2"=> "5" }, "Obj21AttributesObj2"=> { "Obj21AttributesObj2Key1"=> "7" } } }, { "Obj22Id"=> "Id22", "Obj22Version"=> "v1", "Obj22Attributes"=> { "Obj22AttributesObj1"=> { "Obj22AttributesObj1Key1"=> "4", "Obj22AttributesObj1Key2"=> "6" }, "Obj22AttributesObj2"=> { "Obj22AttributesObj2Key1"=> "0" }, "Obj22AttributesArray1"=> [ { "Obj22AttributesArray1Obj1Key1"=> 1 } ] } } ] } ]; say path_to_key('Obj21Id', $struct);

    I've omitted reading in the JSON file into a Perl structure. Use JSON::Tiny or Mojo::JSON or CPANEL::JSON::XS for that.

Re: Obtain the full path of a json key
by tybalt89 (Monsignor) on Jun 22, 2025 at 15:38 UTC

    Short and sweet :)

    #!/usr/bin/perl use strict; # https://perlmonks.org/?node_id=11165436 use warnings; use Path::Tiny; use JSON::PP; sub names { my ($data) = @_; if( 'ARRAY' eq ref $data ) { map names($_), @$data; } elsif( 'HASH' eq ref $data ) { map { my $id = $_; map "$id.$_", names($data->{$_}) } keys %$data; } else { "\n" } } print s/\.$//r for names decode_json path('d.11165436')->slurp; # FIXM +E filename

    Outputs:

    Obj1.Obj11Id Obj1.Obj11Attributes.Obj11AttributesObj2.Obj11AttributesObj2Key1 Obj1.Obj11Attributes.Obj11AttributesObj1.Obj11AttributesObj1Key1 Obj1.Obj11Attributes.Obj11AttributesObj1.Obj11AttributesObj1Key2 Obj1.Obj11Version Obj1.Obj12Attributes.Obj12AttributesArray1.Obj12AttributesArray1Obj1Ke +y1 Obj1.Obj12Attributes.Obj12AttributesObj2.Obj12AttributesObj2Key1 Obj1.Obj12Attributes.Obj12AttributesObj1.Obj12AttributesObj1Key2 Obj1.Obj12Attributes.Obj12AttributesObj1.Obj12AttributesObj1Key1 Obj1.Obj12Version Obj1.Obj12Id Obj2.Obj21Attributes.Obj21AttributesObj2.Obj21AttributesObj2Key1 Obj2.Obj21Attributes.Obj21AttributesObj1.Obj21AttributesObj1Key2 Obj2.Obj21Attributes.Obj21AttributesObj1.Obj21AttributesObj1Key1 Obj2.Obj21Version Obj2.Obj21Id Obj2.Obj22Version Obj2.Obj22Attributes.Obj22AttributesArray1.Obj22AttributesArray1Obj1Ke +y1 Obj2.Obj22Attributes.Obj22AttributesObj2.Obj22AttributesObj2Key1 Obj2.Obj22Attributes.Obj22AttributesObj1.Obj22AttributesObj1Key2 Obj2.Obj22Attributes.Obj22AttributesObj1.Obj22AttributesObj1Key1 Obj2.Obj22Id
      Thank you all for the contributions, I'm installing the missing package Path::Tiny to try all them :)

        You don't really need Path::Tiny. If I didn't have it, I would have done something like this using just standard perl:

        #!/usr/bin/perl use strict; # https://perlmonks.org/?node_id=11165436 use warnings; use JSON::PP; sub names { my ($data) = @_; if( 'ARRAY' eq ref $data ) { map names($_), @$data; } elsif( 'HASH' eq ref $data ) { map { my $id = $_; map "$id.$_", names($data->{$_}) } keys %$data; } else { "\n" } } @ARGV or @ARGV = 'd.11165436.json'; # FIXME filename print map s/\.$//r, names decode_json join '', <>;

        This would allow you to put the JSON file name on the command line, or if you didn't put any arguments on the command line, would run the program against a test file.