How to check that keys in two hashes match and get the corresponding values

NewLondonPerl1 has asked for the wisdom of the Perl Monks concerning the following question:

Hi Perl monks I was hoping you might be able to advise me on how best to do the following. I have been trying for ages but I just cant seem to get it working. I have a large complex data structure that is a hash of hashes of hashes. I am able to parse this but I am struggling with matching up some of key values. Below is a small sample of the data structure at this level that I am trying to parse. As you will see in some of the hashes %filer_device is hash, some of them %filer_volume is a hash, some of them neither is a hash and some of them both are hashes.


"bigstorderv_mpt" => {
        "%export_name" => "/bb/bigstor/derv",
        "%filer_device" => "nydevnfs_derv",
        "%filer_volume" => "/vol/derv"
      },
      "bigstormtg_mpt" => {
        "%export_name" => "/bb/bigstor/mtgmodel",
        "%filer_device" => {
          "\@ridge" => "njdevnfs_mtge",
          "\@west" => "nydevnfs_mtge"
        },
        "%filer_volume" => "/vol/mtge"
      },
     "build10_mpt" => {
        "%export_name" => "/bb/source",
        "%filer_device" => {
          "\@ridge" => "rnap7751-s",
          "\@west" => "nydevnfs_sunbbsource"
        },
        "%filer_volume" => {
          "\@ridge" => "/vol/sunbbsource_c",
          "\@west" => "/vol/sunbbsource"
        }
      },
[download]

I have written this subroutine to try and deal with these nested hashes

 while (my( $kTag, $vTag ) = each %{$nfshashofhashes} ) {
        next if ( ref $vTag ne 'HASH' );
        print "TAG > $kTag\n"; 
        my $default = delete $vTag->{'%default'};

            while (my( $kNFSMNT, $vNFSMNT ) = each %{$vTag} ) {
                next if ($kNFSMNT =~ m/optswt_nfs\b/);
                #print "...$kTag  $kNFSMNT\n";
                my $default = delete $vNFSMNT->{'default'} // delete $
+vNFSMNT->{'%default'};
                while (my($kDEFAULT, $vDEFAULT) = each %{$default}) { 
+    #Get all the defaults key/value pairs for each nfs hash
                    #print "$kNFSMNT:$kDEFAULT $vDEFAULT\n";
                    $MNTOPTS = "$vDEFAULT" if ($kDEFAULT eq '%mount_op
+ts');
                    $MNTUSER = "$vDEFAULT" if ($kDEFAULT eq '%mount_us
+er');
                    $MNTGRP = "$vDEFAULT" if ($kDEFAULT eq '%mount_gro
+up');
                    $MNTACL = "$vDEFAULT" if ($kDEFAULT eq '%mount_acl
+');
                }

                while (my( $kNFSTOKEN, $vNFSTOKEN ) = each %{$vNFSMNT}
+  ) {
                    next if ( $kNFSTOKEN =~ m/\%comment\b/ );
                    #print "*** $kTag $kNFSTOKEN $vNFSTOKEN\n";  #Prin
+ts @dev home7_mpt, @dev home-lnk-mpt etc and value which is a hash co
+ntaing %export_name, %filer_device etc
                
                    $FILERDEVICE = $vNFSTOKEN->{'%filer_device'};
                    $FILERDEVICEHASH = ref $FILERDEVICE ? $FILERDEVICE
+ : { '' => $FILERDEVICE };

                    $FILERVOLUME = $vNFSTOKEN->{'%filer_volume'};
                    $FILERVOLUMEHASH = ref $FILERVOLUME ? $FILERVOLUME
+ : { '' => $FILERVOLUME };

                    $EXPORTFS = $vNFSTOKEN->{'%export_name'};

                    #print "$kTag $kNFSTOKEN $FILERDEVICE $FILERVOLUME
+ $EXPORTFS\n";


                    ###Process %filer_device and %filer_volume. Someti
+mes %filer_device is a hash. Sometimes %filer_volume is a hash. Somet
+imes both are hashes. Sometimes neither are hashes :-(
                    #   
                    my ($FDEVICE, $WESTFDEVICE, $RIDGEFDEVICE, $VOLUME
+, $WESTVOL, $RIDGEVOL);
                    for my $DataCenterFD (keys %{$FILERDEVICEHASH}) {
                        $FDEVICE = $FILERDEVICEHASH->{$DataCenterFD} /
+/ $FILERDEVICEHASH->{''};

                        #print "$DataCenterFD\n";
                             

                        for my $DataCenterVD (keys %{$FILERVOLUMEHASH}
+) {
                            $VOLUME = $FILERVOLUMEHASH->{$DataCenterVD
+} // $FILERVOLUMEHASH->{''};

                            #print "$DataCenterVD\n";

        
                            ##NEARLY WORKING. JUST NEED TO MATCH VOLUM
+E VALUES TO EITHER A @WEST/@RIDGE FILERDEVICE @WEST/@RIDGE
                            #This prints out a line in this format: @d
+ev bigstormtg_mpt @ridge /bb/bigstor/mtgmodel /vol/mtge
                            print "$kTag $kNFSTOKEN $FDEVICE $DataCent
+erFD $EXPORTFS $VOLUME\n";
                        
        
                        }
                    }

                }
            }

    }
}
[download]

The problem I am having is that the filer_device value doesn't match up with what the filer_volume is defined as. So for instance if there is a hash %filer_device with keys @west and @ridge then the filer_device key @west needs to be mapped to the filer_volume @west key value and the same for the @ridge values if these are defined.

Comment on How to check that keys in two hashes match and get the corresponding values Select or Download Code

Replies are listed 'Best First'.
Re: How to check that keys in two hashes match and get the corresponding values by roboticus (Chancellor) on Apr 15, 2013 at 01:49 UTC
NewLondonPerl: I'd suggest rethinking your data structure a bit. The reason you're having a hard time with it is that the data structure isn't regular, so your code is obfuscated by all sorts of special case handling. I couldn't tell what you're wanting to do by looking at your code. Looking at the data, your code and the desired output gave me enough to go on, though. At first, I tried to clean up your code to get it going, but it was just too different from how I would've done it, that I kept getting confused. So I just started from scratch and came up with this: $ cat 1028644.pl #!/usr/bin/perl use strict; use warnings; my $nfshashofhashes = { "bigstorderv_mpt" => { "%export_name" => "/bb/bigstor/derv", "%filer_device" => "nydevnfs_derv", "%filer_volume" => "/vol/derv", }, "bigstormtg_mpt" => { "%export_name" => "/bb/bigstor/mtgmodel", "%filer_device" => { "\@ridge" => "njdevnfs_mtge", "\@west" => "nydevnfs_mtge" }, "%filer_volume" => "/vol/mtge", }, "build10_mpt" => { "%export_name" => "/bb/source", "%filer_device" => { "\@ridge" => "rnap7751-s", "\@west" => "nydevnfs_sunbbsource" }, "%filer_volume" => { "\@ridge" => "/vol/sunbbsource_c", "\@west" => "/vol/sunbbsource" } }, "build11_mpt" => { "%export_name" => "/bb/source", "%filer_device" => { "fridge" => "rnap7751-s", "\@west" => "nydevnfs_sunbbsource", }, "%filer_volume" => { "\@ridge" => "/vol/sunbbsource_c", "\@west" => "/vol/sunbbsource", "qwest" => "/vol/sunquirks", } }, }; while ( my ($MPT, $rMPT) = each %{$nfshashofhashes} ) { die "MPT<$MPT> NOT A HASH!\n" unless ref $rMPT eq 'HASH'; die "MPT<$MPT> WRONG STRUCTURE!\n" unless exists $rMPT->{'%export_ +name'}; my $EXP = $rMPT->{'%export_name'}; # Set default value and hash ref for devices my ($rFILDEV, $DEVDFLT, $t); $t = $rMPT->{'%filer_device'}; if (ref $t eq 'HASH') { $rFILDEV = $t; $DEVDFLT = '????'; } else { $rFILDEV = { }; $DEVDFLT = $t; } # Same for volumes my ($rFILVOL, $VOLDFLT); $t = $rMPT->{'%filer_volume'}; if (ref $t eq 'HASH') { $rFILVOL = $t; $VOLDFLT = '????'; } else { $rFILVOL = { }; $VOLDFLT = $t; } # Match up the VOL and DEV entries in the hash. Missing values fo +r # DEV and VOL will be filled in with the DFLT values as needed. my %MATCHUP; $MATCHUP{$_}{VOL} = $rFILVOL->{$_} for keys %$rFILVOL; $MATCHUP{$_}{DEV} = $rFILDEV->{$_} for keys %$rFILDEV; # If both were scalars, MATCHUP is empty, so make a single matchin +g record $MATCHUP{'BAR'} = { DEV=>$DEVDFLT, VOL=>$VOLDFLT } if ! keys %MATC +HUP; # And our final results... for (keys %MATCHUP) { my $DEV = $MATCHUP{$_}{DEV} // $DEVDFLT; my $VOL = $MATCHUP{$_}{VOL} // $VOLDFLT; printf "%-20.20s %-20.20s %-20.20s %-20.20s\n", $MPT, $DEV, $VOL, $EXP; } } $ perl 1028644.pl bigstorderv_mpt nydevnfs_derv /vol/derv /bb/big +stor/derv build11_mpt ???? /vol/sunbbsource_c /bb/sou +rce build11_mpt rnap7751-s ???? /bb/sou +rce build11_mpt ???? /vol/sunquirks /bb/sou +rce build11_mpt nydevnfs_sunbbsource /vol/sunbbsource /bb/sou +rce build10_mpt rnap7751-s /vol/sunbbsource_c /bb/sou +rce build10_mpt nydevnfs_sunbbsource /vol/sunbbsource /bb/sou +rce bigstormtg_mpt njdevnfs_mtge /vol/mtge /bb/big +stor/mtgmodel bigstormtg_mpt nydevnfs_mtge /vol/mtge /bb/big +stor/mtgmodel [download] I think that'll do it for you. You didn't specify what you were expecting when you had two hashes with mismatched keys, so I added "build11_mpt" to demonstrate it. Feel free to change it as you need it. Since I had trouble wrapping my head around your code, I fully expect you might have a little difficulty with mine, so feel free to ask questions. ...roboticus When your only tool is a hammer, all problems look like your thumb.	[reply] [d/l]
Re^2: How to check that keys in two hashes match and get the corresponding values by NewLondonPerl1 (Acolyte) on Apr 15, 2013 at 03:35 UTC
Thanks ever so much for your help Roboticus. I have tried this out and this is now matching up these key values perfectly. I just have one question though. Maybe it because I dont fully understand all of your code but when I run this its seems to print out lines more than once. This seems to happen when either %filer_device is a hash or %filer_volume is a hash or both are hashes. So for example in the following section: `"rtsys_mpt": { "%export_name": "/bb/rtsys", "%filer_device": { "@west": "nydevnfs_rtsysgit", "@ridge": "njdevnfs_rtsysgit" }, "%filer_volume": "/vol/rtsysgit" }` [download] When I run your code on this and just purely print out the volume and device values i get the lines duplicated like so: `njdevnfs_rtsysgit = /vol/rtsysgit nydevnfs_rtsysgit = /vol/rtsysgit njdevnfs_rtsysgit = /vol/rtsysgit nydevnfs_rtsysgit = /vol/rtsysgit` [download] All that I need is this: `njdevnfs_rtsysgit = /vol/rtsysgit nydevnfs_rtsysgit = /vol/rtsysgit` [download] But for some reason its duplicated? I am not too sure why? and in this section where %filer_device and %filer_volume are both hashes it outputs the same lines 3 times: `"build18_mpt": { "%export_name": "/bb/mobile", "%filer_device": { "@west": "nydevnfs_mob_build", "@ridge": "rnap2113-s" }, "%filer_volume": { "@west": "/vol/mob_build", "@ridge": "/vol/mob_build_c" } },` [download] `nydevnfs_mob_build = /vol/mob_build rnap2113-s = /vol/mob_build_c nydevnfs_mob_build = /vol/mob_build rnap2113-s = /vol/mob_build_c nydevnfs_mob_build = /vol/mob_build rnap2113-s = /vol/mob_build_c` [download] All i need is the first two lines. The others are duplicates: `nydevnfs_mob_build = /vol/mob_build rnap2113-s = /vol/mob_build_c` [download] Do you know why this is happening please?	[reply] [d/l] [select]
Re^3: How to check that keys in two hashes match and get the corresponding values by roboticus (Chancellor) on Apr 15, 2013 at 12:43 UTC
NewLondonPerl1: I'm not seeing the problem, so I guess you made some change that you haven't mentioned. Without making any changes other than adding your new test case: `. . . snip . . . my $nfshashofhashes = { "rtsys_mpt" => { '%export_name' => '/bb/rtsys', '%filer_device' => { '@west' => 'nydevnfs_rtsysgit', '@ridge' => 'njdevnfs_rtsysgit', }, '%filer_volume' => '/vol/rtsysgit', }, "bigstorderv_mpt" => { . . . snip . . .` [download] I get the following, which seems to be what you're wanting: $ perl 1028644.pl bigstorderv_mpt nydevnfs_derv /vol/derv /bb/big +stor/derv build11_mpt ???? /vol/sunbbsource_c /bb/sou +rce build11_mpt rnap7751-s ???? /bb/sou +rce build11_mpt ???? /vol/sunquirks /bb/sou +rce build11_mpt nydevnfs_sunbbsource /vol/sunbbsource /bb/sou +rce build10_mpt rnap7751-s /vol/sunbbsource_c /bb/sou +rce build10_mpt nydevnfs_sunbbsource /vol/sunbbsource /bb/sou +rce rtsys_mpt njdevnfs_rtsysgit /vol/rtsysgit /bb/rts +ys rtsys_mpt nydevnfs_rtsysgit /vol/rtsysgit /bb/rts +ys bigstormtg_mpt njdevnfs_mtge /vol/mtge /bb/big +stor/mtgmodel bigstormtg_mpt nydevnfs_mtge /vol/mtge /bb/big +stor/mtgmodel [download] ...roboticus When your only tool is a hammer, all problems look like your thumb.	[reply] [d/l] [select]
Re: How to check that keys in two hashes match and get the corresponding values by gnosti (Chaplain) on Apr 15, 2013 at 08:05 UTC
Speaking as someone who has probably spent 3/4 of his development time or more in debugging, I would suggest not using sigils $%@ in your keys, i.e. use `filer_device` instead of `%filer_device`. Perhaps I'm just narrow-minded, but it seems confusing, and not my idea of "Perlish". I admit this isn't speaking to your problem.	[reply] [d/l] [select]
Re: How to check that keys in two hashes match and get the corresponding values by NewLondonPerl1 (Acolyte) on Apr 14, 2013 at 20:29 UTC
So essentially using the above small hash examples this is the output I am looking for: `@dev bigstorderv_mpt nydevnfs_derv /vol/derv /bb/bigstor/derv @dev bigstormtg_mpt njdevnfs_mtge /vol/mtge /bb/bigstor/mtgmodel @dev bigstormtg_mpt nydevnfs_mtge /vol/mtge /bb/bigstor/mtgmodel @dev build10_mpt rnap7751-s /vol/sunbbsource_c /bb/source @dev build10_mpt nydevnfs_sunbbsource /vol/sunbbsource /bb/source` [download] I am getting very similar output except I cant seem to get the value of the %volume_device to match the value set for the %filer_device so instead of getting the output above I get some lines are written twice as my sub routine doesn't know which volume_device value to map to the filer_device value so it prints the lines twice with the value for the volume device different on each line Please can you help me with what I am doing wrong here?	[reply] [d/l]
Re: How to check that keys in two hashes match and get the corresponding values by aitap (Curate) on Apr 15, 2013 at 17:43 UTC
`while (my ($name, $hash) = each %nfs) { for my $devloc (ref $hash->{'%filer_device'} ? keys %{$hash->{'%fi +ler_device'}} : '%filer_device') { # hack here my $device = (ref $hash->{'%filer_device'} ? $hash->{'%filer_d +evice'} : $hash)->{$devloc}; # and here my $volume = ref $hash->{'%filer_volume'} ? $hash->{'%filer_vo +lume'}{$devloc} : $hash->{'%filer_volume'}; print "\@dev $name $device $volume $hash->{'%export_name'}\n"; } }` [download] This code uses a small hack providing itself a fake hash and hash key when '%filer_device' value is not a hash reference. The downside of it is lack of data structure checking, so the real solution should be more verbose (and probably use Params::Validate, Data::Validator, Data::Diver or Data::DPath).	[reply] [d/l]