How to get contents from an array inside a hash

gingera2z has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to build a utility that will create a character set, based on a pre-defined record layout. The code loads the layout into a hash, updates the character set for each field as it reads the input file, and then it's supposed to print the character set to a results file. Data::Dumper shows the expected character sets, but the printed output is blank for the character sets.

I have removed several comments and display statements to keep the size down, but wasn't sure if my problem might be higher in the code so have included it all. I believe the issue is in the "Format Printed layout" section.

Layout file example:

StudentLastName,180,30,Y,,Y
StudentFirstName,210,30,N,,Y
StudentMiddleName,240,30,N,,
DateOfBirth,270,8,Y,,Y
Gender,278,2,Y,,Y
Grade,280,3,Y,,
[download]

Input file example:

 111E2000                                                             
+           111E2000                                                  
+                    201457660112567001  EP015F                       
+ SSHS                                                        12232006
+01120      00   0    000   0         111        0   0                
+      sumsoc12S   EPEN SS HS                                         
+   NNNNNNNNNN                           0000091323                   
+   Y
 111E2000                                                             
+           111E2000                                                  
+                    201457660212567002  EP025F                       
+ SSHS                                                        12232006
+01120      00   0    000   0         111        0   0                
+      sumsoc12S   EPEN SS HS                                         
+   NNNNNNNNNN                           0000091324                   
+   Y
 111E2000                                                             
+           111E2000                                                  
+                    201457660312567003  EP035F                       
+ SSHS                                                        12232006
+01120      00   0    000   0         111        0   0                
+      sumsoc12S   EPEN SS HS                                         
+   NNNNNNNNNN                           0000091325                   
+   Y
[download]

And the code...

#*************************************
#  Define external utilities
#*************************************
use strict;
#use warnings;
#use diagnostics;

use List::MoreUtils qw{ any };
use Data::Dumper qw(Dumper);

#*************************************
#  Assign the variables with the value passed to program
#*************************************
    my $Layout     = $ARGV[0];
    my $File1     = $ARGV[1];    
    my $Results    = $ARGV[2];


#*************************************
#     Initialize Global Variables
#*************************************
    my $cnt_records_read = 0;
    my $null = "";

#*************************************
#     Open Files. 
#*************************************
    my ($layoutFile, $inputFile, $resultsFile);
    
    if (-e $Layout) 
    { 
      open($layoutFile, "<", "$Layout") or die("Could not open Layout 
+file: $!\n");
    } 
    
    if (-e $File1) 
    { 
      open($inputFile, "<", "$File1") or die("Could not open File 1. $
+!\n");
    } 
    
    open($resultsFile, ">", "$Results") or die("Could not open Results
+ File. Verify user has permissions for the results folder location. $
+!\n");


#*************************************
#    Create hash of layout file 
#*************************************
    my $cnt_all_fields = 0;
    my $cnt_omitted_fields = 0;
    my %layout=(); my $layout;
    my $layout_key = $null;
    my @layout_value = "";
    my $layout_value = $null;
    my @chars=();
    
    while (my $line1=<$layoutFile>) 
    {
        chomp $line1;
        my @fields = split("," , $line1);
        
        if ( $fields[3] eq "Y" ) {
            $cnt_all_fields++;
            my $layout_key = $fields[0];
            my $layout_value = {
                 'Field Name' => $fields[0] 
                ,'Position' => $fields[1] - 1 
                ,'Length' => $fields[2] - 0
                ,'Characters' => [ @chars ]
                };
                             
            $layout{ $layout_key } = [ $layout_value ];
        } else {
            $cnt_omitted_fields++;
        }
    }    
#    print Dumper %layout; 

#*************************************
#    Update Characters Array in Layout Hash
#*************************************
    my $cnt_records_read_file1=0;
    my $char;
    my $chars;
    my @fld_detail;
    my @fld_chars=();
    
    while ( my $line1 = <$inputFile> ) {
       $cnt_records_read_file1++;
       chomp($line1);        
       
       foreach my $name ( keys %layout ) {
        foreach my $fld_detail ( @{ $layout{$name} } ) {
             
             my $fld_name     = $fld_detail->{'Field Name'};
            my $fld_position = $fld_detail->{'Position'};
            my $fld_length   = $fld_detail->{'Length'};
            my @fld_chars    = @{ $layout{$name}}{'Characters'};
            
            my $field1 = substr($line1,$fld_position,$fld_length);
            
            for(my $pos = 0; $pos < $fld_length; $pos++) {
                my $char = substr($field1,$pos,1);
                                
                if (any { $_ eq $char } @fld_chars) {
                } else {
                    push @fld_chars, $char;
                     $layout_key = $fld_name; 
                    $layout_value = {
                     'Field Name' => $fld_name
                    ,'Position' => $fld_position 
                    ,'Length' => $fld_length
                    ,'Characters' => [ @fld_chars ]
                    };
                    $layout{ $layout_key } = [ $layout_value ];    
                }
                
            }
        }
       }
    }
    print Dumper \%layout; 
    
#*************************************
#    Format printed layout
#*************************************
       foreach my $name ( keys %layout ) {
        foreach my $fld_detail ( @{ $layout{$name} } ) {
             
             my $fld_name     = $fld_detail->{'Field Name'};
            my $fld_position = $fld_detail->{'Position'};
            my $fld_length   = $fld_detail->{'Length'};
            my @fld_chars    = @{$layout{$name}}{'Characters'};

        print @fld_chars;
        
        $fld_name = sprintf("%-30s",$fld_name);
            $fld_position = sprintf("%8s",$fld_position);
            $fld_length = sprintf("%6s",$fld_length);
            
            print $resultsFile "$fld_position $fld_length $fld_name @f
+ld_chars \n";
        }
        }

#*************************************
#     Close Files
#*************************************
    close($layoutFile);
    close($inputFile);
    close($resultsFile);
[download]

Dumper output:

$VAR1 = {
          'StudentLastName' => [
                                 {
                                   'Length' => 30,
                                   'Position' => 179,
                                   'Field Name' => 'StudentLastName',
                                   'Characters' => [
                                                     undef,
                                                     'E',
                                                     'P',
                                                     '1',
                                                     '9',
                                                     '5',
                                                     'F',
                                                     ' '
                                                   ]
                                 }
                               ],
          'SASID' => [
                       {
                         'Length' => 10,
                         'Position' => 159,
                         'Field Name' => 'SASID',
                         'Characters' => [
                                           undef,
                                           '2',
                                           '0',
                                           '1',
                                           '4',
                                           '5',
                                           '7',
                                           '6',
                                           '9'
                                         ]
                       }
                     ],
          'Grade' => [
                       {
                         'Length' => 3,
                         'Position' => 279,
                         'Field Name' => 'Grade',
                         'Characters' => [
                                           undef,
                                           '1',
                                           '2',
                                           '0'
                                         ]
                       }
                     ],
          'Gender' => [
                        {
                          'Length' => 2,
                          'Position' => 277,
                          'Field Name' => 'Gender',
                          'Characters' => [
                                            undef,
                                            '0',
                                            '1'
                                          ]
                        }
                      ],
          'DateOfBirth' => [
                             {
                               'Length' => 8,
                               'Position' => 269,
                               'Field Name' => 'DateOfBirth',
                               'Characters' => [
                                                 undef,
                                                 '1',
                                                 '2',
                                                 '3',
                                                 '0',
                                                 '6'
                                               ]
                             }
                           ]
        };
[download]

And sample warnings. There are many variations of @fld_chars is undefined notifications.

Pseudo-hashes are deprecated at procFreq2.pl line 136, <$inputFile> li
+ne 20.
Use of uninitialized value in string eq at procFreq2.pl line 143, <$in
+putFile> line 20.

Pseudo-hashes are deprecated at procFreq2.pl line 186, <$inputFile> li
+ne 20.
Use of uninitialized value in print at procFreq2.pl line 188, <$inputF
+ile> line 20.
[download]

Comment on How to get contents from an array inside a hash Select or Download Code

Replies are listed 'Best First'.
Re: How to get contents from an array inside a hash by GrandFather (Saint) on Dec 05, 2014 at 23:37 UTC
There are quite a few bugs happily tucked away in that code. A few coding techniques will help find many of them: Remove cruft. If you don't use the contents of a variable remove it ($cnt_all_fields etc.). That doesn't fix bugs, but it gives them fewer places to hide. Declare variables in the smallest scope you can. Don't use globals if you can possibly avoid them. @chars is an interesting case in point and probably relates to a bug. Use named variables rather than arrays. That doesn't fix bugs either, but makes code intent more obvious. @fields is the first example - use `my ($key, $pos, $len, $use) = split ',', $line;` instead. Don't invent your own variant of undef. Perl already has its own version and can help spot bugs for you if you use it with warnings on. Variables are undef by default and arrays and hashes are empty by default so you don't need to set them to that state. See the various layout variables for example. Don't reuse variable names for different variable types - that way madness lies. Example: @layout_value and $layout_value. Perl helps catch more bugs if you avoid overloaded names and the code is easier to understand. Perl is a real wizz at string handling. You should almost never need to loop over a string and pull characters out one by one! It's slow and harder than it needs to be to understand. Ok, that's enough to be going on with. Here's a somewhat "cleaned up" version of your code. It probably still doesn't work as you want, but at least a few gremlins have been shaken loose: use strict; use warnings; use List::MoreUtils qw{ any }; my $cnt_records_read = 0; my $layout = <<'LAY'; StudentLastName,180,30,Y,,Y StudentFirstName,210,30,N,,Y StudentMiddleName,240,30,N,, DateOfBirth,270,8,Y,,Y Gender,278,2,Y,,Y Grade,280,3,Y,, LAY my %layout; my @params = qw(Name Position Length); open my $layoutFile, '<', \$layout; while (defined (my $line = <$layoutFile>)) { chomp $line; my ($name, $pos, $len, $use) = split ",", $line; next if $use ne "Y"; @{$layout{$name}}{@params} = ($name, $pos - 1, $len); } close $layoutFile; my @records; my @fieldNames = sort keys %layout; while (my $line = <DATA>) { push @records, {}; for my $param (@fieldNames) { my $field = $layout{$param}; my $data = substr $line, $field->{Position}, $field->{Length}; $records[-1]{$param} = $data; } } for my $record (@records) { for my $name (@fieldNames) { printf "%-30s %8s %6s %s\n", @{$layout{$name}}{@params}, $record->{$name}; } } __DATA__ 111E2000 + 111E2000 + 201457660112567001 EP015F + SSHS 12232006 +01120 00 0 000 0 111 0 0 + sumsoc12S EPEN SS HS + NNNNNNNNNN 0000091323 + Y 111E2000 + 111E2000 + 201457660212567002 EP025F + SSHS 12232006 +01120 00 0 000 0 111 0 0 + sumsoc12S EPEN SS HS + NNNNNNNNNN 0000091324 + Y 111E2000 + 111E2000 + 201457660312567003 EP035F + SSHS 12232006 +01120 00 0 000 0 111 0 0 + sumsoc12S EPEN SS HS + NNNNNNNNNN 0000091325 + Y [download] which prints: `DateOfBirth 269 8 12232006 Gender 277 2 01 Grade 279 3 120 StudentLastName 179 30 EP015F + DateOfBirth 269 8 12232006 Gender 277 2 01 Grade 279 3 120 StudentLastName 179 30 EP025F + DateOfBirth 269 8 12232006 Gender 277 2 01 Grade 279 3 120 StudentLastName 179 30 EP035F` [download] Perl is the programming world's equivalent of English	[reply] [d/l] [select]
Re: How to get contents from an array inside a hash by poj (Abbot) on Dec 05, 2014 at 21:24 UTC
Treat `Characters` the same as the other fields but as an array reference `#!perl use strict; use Data::Dump 'pp'; my %layout=( 'key' => [{'Characters' => [ 'A','B','C' ]},], ); pp \%layout; foreach my $fld_detail ( @{ $layout{'key'} } ) { my $ar = $fld_detail->{'Characters'}; print @$ar; }` [download] poj	[reply] [d/l] [select]
Re^2: How to get contents from an array inside a hash by gingera2z (Initiate) on Dec 05, 2014 at 21:35 UTC
That was what I needed. Thank you!	[reply]
Re: How to get contents from an array inside a hash by toolic (Bishop) on Dec 05, 2014 at 20:50 UTC
Do you get any warning messages when you uncomment warnings? Post a small sample of your `print Dumper(\%layout);` output (note the backslash). Add more print's to identifiy where things go wrong. http://sscce.org	[reply] [d/l]
Re: How to get contents from an array inside a hash by GrandFather (Saint) on Dec 05, 2014 at 21:08 UTC
Where is the sample layout file data? Why one big blob of code instead of subs that can be tested individually? Why oh why did you turn off warnings! Perl is the programming world's equivalent of English	[reply]
Re^2: How to get contents from an array inside a hash by gingera2z (Initiate) on Dec 05, 2014 at 21:18 UTC
The sample layout file is at the top of the listing. I'll update and include some sample input data. I'm currently coding without subroutines because I'm still learning. I know the code could be more elegant, and definitely more efficient, but I can't start by banging my head against the keyboard and praying it works. :) Warnings is off because that's a shop standard for code that gets pushed to production. I simply hadn't included it when posting.	[reply]
Re^3: How to get contents from an array inside a hash by karlgoethebier (Abbot) on Dec 05, 2014 at 21:46 UTC
"...banging my head against the keyboard and praying..." Stop praying and head banging; it doesn't help. Start to break it down into smaller pieces instead - at least four subs ;-) A sketch in a hurry: `get_options() # use GetOpt::Long` `create_hash()` `update_chars()` `format_layout()` ..or so. Update: I forgot to update, sorry. Consider also to put your stuff in a module. Please see perlmod, perlnewmod and Simple Module Tutorial how to accomplish this. And it is a good idea to study some sources on cpan. Regards, Karl �The Crux of the Biscuit is the Apostrophe�	[reply] [d/l] [select]
Re: How to get contents from an array inside a hash by AnomalousMonk (Archbishop) on Dec 05, 2014 at 21:42 UTC
Don't really know what you're doing, but maybe check perldsc (Perl Data Structures Cookbook) for more insight into your problem.	[reply]