Flip76 has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I'm stucked and can't find any solution. Since 3 days I am researching all kind of forums/documentation but I can't get any hint.

Here is my problem:

I'm requesting data from a REST-API and I'm receiving JSON-Format. Now I want to build a Dereference (e. g. $item->{meta}->{created}) dynamically but it does only work for the 1st level ($item->{userName}).

Maybe you can help relieving my headache?

Regards,
Flip

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # Code Sniplet #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ my $data_structure = decode_json($client->responseContent()); print Dumper($data_structure); # Output of print Dumper($data_structure); $VAR1 = { 'Resources' => [ 'displayName' => 'User 1', 'userName' => 'user1@test.com', 'id' => '1234567', 'meta' => { 'created' => '2018-04-16T11:5 +7:19.376Z' } }, { 'displayName' => 'User 2', 'userName' => 'user2@test.com', 'id' => '1234568', 'meta' => { 'created' => '2018-04-16T11:5 +9:27.111Z' } }, { 'displayName' => 'User 3', 'userName' => 'user3@test.com', 'id' => '12345679', 'meta' => { 'created' => '2018-11-21T14:4 +9:33.821Z' } }, ], 'totalResults' => 3, 'itemsPerPage' => 50, 'startIndex' => 1, 'schemas' => [ 'urn:ietf:params:scim:api:messages:2.0:ListRe +sponse' ] }; # Normally @elements is filled by entries from a config-file # but for make it easy, I set the values here my @elements; push(@elements,"userName"); push(@elements,"displayName"); push(@elements,"meta.created"); foreach $item ( @{ $data_structure->{Resources} } ) { foreach $element (@elements) { # Convert a . to }->{ in order for meta.created to look like m +eta}->{created $element =~ s/\./}->{/g; # This works for all elements except of meta}->{created $deref = $item->{$element}; print "Convert: \$item->{" . $element . "}\t\t\t" . $deref . +"\n"; } # But this works deref = $item->{meta}->{created}; print "When use direct: \$item->{meta}->{created}\t" . $deref . "\ +n"; print "\n"; } #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # Output: #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Convert: $item->{userName} user1@test.com Convert: $item->{displayName} User 1 Use of uninitialized value $deref in concatenation (.) or string at /h +ome/dczesak/cis4oci/cis4oci.pl line 291. Convert: $item->{meta}->{created} When use direct: $item->{meta}->{created} 2018-04-16T11:57:19.37 +6Z Convert: $item->{userName} user2@test.com Convert: $item->{displayName} User 2 Use of uninitialized value $deref in concatenation (.) or string at /h +ome/dczesak/cis4oci/cis4oci.pl line 291. Convert: $item->{meta}->{created} When use direct: $item->{meta}->{created} 2018-04-16T11:59:27.11 +1Z Convert: $item->{userName} user3@test.com Convert: $item->{displayName} User 3 Use of uninitialized value $deref in concatenation (.) or string at /h +ome/dczesak/cis4oci/cis4oci.pl line 291. Convert: $item->{meta}->{created} When use direct: $item->{meta}->{created} 2018-11-21T14:49:33.82 +1Z

2018-12-08 Athanasius changed <pre> to <p> tags around text

Replies are listed 'Best First'.
Re: Dynamic Dereferencing
by haukex (Archbishop) on Dec 07, 2018 at 11:32 UTC

    You may be interested in Data::Diver. I also showed some custom "diver" type code here and here. Update: There's also Data::DPath or Data::Path, I haven't used these myself but they seem worth a look.

      Thanks, I will give it a try

        If you know your data then maybe a simplified version of the code in Data::Diver is all you need.

        #!perl use strict; use JSON qw 'decode_json'; my $json = join '',<DATA>; $json =~ s/\n//g; my $data_structure = decode_json($json); my @elements = qw(userName displayName meta.created ); for my $item ( @{ $data_structure->{Resources} } ){ for my $e (@elements){ my $ref = $item; my @keys = split '\.',$e; while (@keys){ my $key = shift @keys; $ref = $ref->{$key}; } printf "$e = %s\n",$ref; } print "\n"; } __DATA__ {"Resources":[ {"id":"1234567","userName":"user1@test.com", "meta":{"created":"2018-04-16T11:57:19.376Z"},"displayName":"User 1"}, {"id":"1234568","userName":"user2@test.com", "meta":{"created":"2018-04-16T11:59:27.111Z"},"displayName":"User 2"}, {"id":"12345679","userName":"user3@test.com", "meta":{"created":"2018-11-21T14:49:33.821Z"},"displayName":"User 3"} ],"schemas":["urn:ietf:params:scim:api:messages:2.0:ListResponse"], "startIndex":1,"itemsPerPage":50,"totalResults":3}
        poj
Re: Dynamic Dereferencing
by 1nickt (Canon) on Dec 07, 2018 at 12:08 UTC

    Hi, when trying to access nested elements of data structures using key "paths" as you have, I usually turn to a JSON pointer. I find Mojo::JSON::Pointer easiest to use. In the following demonstration I am being lazy and returning your Dumper dump from a sub and then converting it back to JSON. If you are using LWP::UserAgent's decoded_content() you will already have the structure in $data below.

    use v5.014; use JSON; use Mojo::JSON::Pointer; # reconstructing a JSON response from your Dump my $json = to_json( get_response() ); # your client request gets this # converting your API response to a Perl struct my $data = from_json( $json ); # your client should provide this # converting the elements to JSON pointer paths my @elements = map {s!\.!/!gr} (qw/ userName displayName meta.created +/); for my $item ( @{ $data->{Resources} } ) { my $pointer = Mojo::JSON::Pointer->new( $item ); foreach my $element (@elements) { say "$element: " . $pointer->get('/' . $element); } say ''; } sub get_response { return { 'Resources' => [{ 'displayName' => 'User 1', 'userName' => 'user1@test.com', 'id' => '1234567', 'meta' => { 'created' => '2018-04-16T11:5 +7:19.376Z' } }, { 'displayName' => 'User 2', 'userName' => 'user2@test.com', 'id' => '1234568', 'meta' => { 'created' => '2018-04-16T11:5 +9:27.111Z' } }, { 'displayName' => 'User 3', 'userName' => 'user3@test.com', 'id' => '12345679', 'meta' => { 'created' => '2018-11-21T14:4 +9:33.821Z' } }, ], 'totalResults' => 3, 'itemsPerPage' => 50, 'startIndex' => 1, 'schemas' => [ 'urn:ietf:params:scim:api:messages:2.0:ListRe +sponse' ] }; } __END__
    Output:
    perl 1226873.pl userName: user1@test.com displayName: User 1 meta/created: 2018-04-16T11:57:19.376Z userName: user2@test.com displayName: User 2 meta/created: 2018-04-16T11:59:27.111Z userName: user3@test.com displayName: User 3 meta/created: 2018-11-21T14:49:33.821Z

    Hope this helps!


    The way forward always starts with a minimal test.
      Thanks a lot for the nice sample!

      One more question:
      # converting the elements to JSON pointer paths my @elements = map {s!\.!/!gr} (qw/ userName displayName meta.created +/);
      qw does not support interpolation right? So I can't use a string there too :-(
      Is there a way around to dynamically change the values there?
      Background: The values are coming from a config-file. So if I add a new entry like "meta.user" in the config, it should work without touching the code...
        Ups sorry, I got it:
        my @elements = map {s!\.!/!gr} (split(' ', $API{$action . '.elements'} +));
        Thanks a lot for your help!
Re: Dynamic Dereferencing
by bliako (Abbot) on Dec 07, 2018 at 11:32 UTC

    you are missing a dollar, line 290

    otherwise, $VAR1 is what you call a dereference, right?

      Oh sorry, line 290 is wrong as I used the output from my script and not from the snipplet here - actually this line is meant:

      $deref = $item->{$element};

        here then:

        # But this works deref = $item->{meta}->{created};
Re: Dynamic Dereferencing
by markong (Pilgrim) on Dec 07, 2018 at 12:10 UTC
    # Convert a . to }->{ in order for meta.created to look like meta}->{created $element =~ s/\./}->{/g; # This works for all elements except of meta}->{created $deref = $item->{$element}; print "Convert: \$item->{" . $element . "}\t\t\t" . $deref . "\n"; }

    I think you are trying to achieve something with a sort of "meta-coding" which is quite dangerous: when you write $item->{$element} you're try to access the hash $item value with a key like "...}->{..." which is why you then get that error, because there isn't such a key!

    If you really want to do such a sort of meta-coding, then you need to write for instance  "$item->{".$element."}" and then you can try to eval() that thing. Why instead not trying one of the modules recommended by haukex ?

      String eval is not to be used . especially not as a substitute for functions and loops for dereference

        ?

        I was replying to the OP on why he had that error and the only reasonable way to correct that code. I never recommended eval-ing concatenated strings as a good practice. The opposite as a matter of facts!

        Eval()-ing has, of course, its use cases, but as I said, it's dangerous in this particular case. One of the reasons of why I encouraged using some of the mentioned CPAN alternatives, if possible, to navigate nested structures.