DreamT has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

Let's say that I have a data structure from an external source, stored as a hash ref. For example 
$external_structure = {
    'name' => 'Test product',
    'id' => 'ABC123',
    'data' => {
                 'enabled' => '1',
                 'internal_id' => '',
                 'urls' => {
                              'main_url' => ''
                            },
                 'price' => '1000',
                 'unit' => 'ST',
                 'unit_info' => {
                                   'qty' => '1',
                                   'gtin' => '1234567890'
                                 }
               }
        };

[download]

Now, say that I want to map the fields to another structure (to be put in a database/textfile or so later on). I could do it in the following way:
my $internal_structure = {};

$internal_structure->{'product_id'} = $external_structure->{'id'};
$internal_structure->{'price_excl_vat'} = $external_structure->{'data'
+}->{'price'};
$internal_structure->{'qty'} = $external_structure->{'data'}->{'unit_i
+nfo'}->{'qty'};
# etc...

[download]
However, let's say that an end user wants to control the mappings, without having knowledge about or access to the source code. In theory, the mappings could be put in a textfile/CRM system or similar, like 
product_id;id
price_excl_vat;data#price
qty;data#unit_info#qty

[download]
(There are of course better ways to do it, but just to explain the principle)

But, how can I then apply the mapping to the code? I have ideas on how to do it, but none of them seems elegant or foolproof...I would really like to be inspired on this one, so please hit me with your best ideas! There is no rule regarding how the mapping should look like, more than that is should be stored outside of the source code. And, there is no limit on how many levels the source structure has, so the solution can't be locked to a certain number of levels.

Also note that I unfortunately can't use any external module (like Data::Diver), I need a "core Perl"solution.

Replies are listed 'Best First'.
Re: Mapping data structures through external source
by choroba (Cardinal) on Aug 02, 2019 at 14:14 UTC
    That's exactly the case where Data::Diver is helpful! 
    #!/usr/bin/perl
use warnings;
use strict;

use Data::Diver qw{ Dive };

my $external_structure = { ... };

my %format = (product_id     => ['id'],
              price_excl_vat => [qw[ data price ]],
              unit_qty       => [qw[ data unit_info qty ]]);  # (1)

my $internal_structure = {};
for my $key (keys %format) {
    $internal_structure->{$key}
        = Dive($external_structure, @{ $format{$key} });
}

use Data::Dumper; print Dumper $internal_structure;

    [download]

    (1) Note that the second element of the path is unit_info, not just unit.

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
[reply]
[d/l]
[select]
      That's exactly the case where Data::Diver is helpful!

      This is my first exposure to Data::Diver, and I'm impressed and happy to put this in my bag of various scripts:

      $ ./1.diver.pl 
$VAR1 = {
          'product_id' => 'ABC123',
          'unit_qty' => '1',
          'price_excl_vat' => '1000'
        };
$ cat 1.diver.pl

      [download]
      #!/usr/bin/perl
use 5.016;
use warnings;
use Data::Diver qw{ Dive };
use Data::Dumper;

my $external_structure = {
  'name' => 'Test product',
  'id'   => 'ABC123',
  'data' => {
    'enabled'     => '1',
    'internal_id' => '',
    'urls'        => {
      'main_url' => ''
    },
    'price'     => '1000',
    'unit'      => 'ST',
    'unit_info' => {
      'qty'  => '1',
      'gtin' => '1234567890'
    }
  }
};

my %format = (
  product_id     => ['id'],
  price_excl_vat => [qw[ data price ]],
  unit_qty       => [qw[ data unit_info qty ]]
);    # (1)

my $internal_structure = {};
for my $key ( keys %format ) {
  $internal_structure->{$key} =
    Dive( $external_structure, @{ $format{$key} } );
}

print Dumper $internal_structure;

__END__
$

      [download]
      Note that the second element of the path is unit_info, not just unit.

      Well noted. This points out the problem of whether to "fix" the original post. If OP does, then both of our posts might need "fixing" too. My preference is to leave the warts on the original post.

[reply]
[d/l]
[select]
      Thank you for your reply (and thank you for the note regarding the structure error - Fixed now).
      However, I'm in a position where I can't install Data::Diver (or any other external module for that matter). Do you have any "core Perl" solutions to suggest?
[reply]
        However, I'm in a position where I can't install Data::Diver (or any other external module for that matter).

        Yes, even you can use CPAN! I strongly suggest to take some time to look into e.g. local::lib or maybe App::FatPacker.

        Having said that, I've written some custom "diver" type code here and here.

[reply]
Re: Mapping data structures through external source
by Fletch (Bishop) on Aug 02, 2019 at 20:01 UTC

    TMTOWTDI sometimes not even involving (much) perl; if the inbound data were JSON I'd look at jq. 

    #!/usr/bin/env perl
use 5.018;
use JSON::XS ();

my $json = JSON::XS->new->pretty(1)->ascii(1);

my $external_structure = {
    'name' => 'Test product',
    'id'   => 'ABC123',
    'data' => {
        'enabled'     => '1',
        'internal_id' => '',
        'urls'        => { 'main_url' => '' },
        'price'       => '1000',
        'unit'        => 'ST',
        'unit_info'   => {
            'qty'  => '1',
            'gtin' => '1234567890'
        }
    }
};

say $json->encode($external_structure);

exit 0;

__END__

$ perl splonk.plx | 
    jq '{ product_id: .id, price_excl_vat: .data.price, unit_qty: .dat
+a.unit_info.qty }'
{
  "product_id": "ABC123",
  "price_excl_vat": "1000",
  "unit_qty": "1"
}

    [download]

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

[reply]
[d/l]

Back to Seekers of Perl Wisdom