Pharazon has asked for the wisdom of the Perl Monks concerning the following question:

Okay so to start with I have looked at several threads for how to handle undefined values in arrays as well as read up on the differences between undefined, empty, etc. While I am not a perl expert by any means I am still unable to come up with a solution to my problem even after all of this and could really use some of your monkish brilliance.

I have a file that is formatted (not fixed position) that will always have 58 lines to one record. I am reading in up to the line break character in the data (\014) and then using split to break the record into lines. Then I am passing the array to a function that will pull and format information according to my needs and then build an array for return. The problem I am having is that I have some lines that only contain a newline in the data and (I assume) these are coming is as undefined. While they don't seem to be causing the logic any issues I am getting a massive number of uninitialized warnings and I don't like programming in a way that causes warnings regardless if their harmless or not, so I don't really want to just turn them off. Below is the code that I have written so far. I don't expect that is as efficient as it can be or as succinct but as I have just started learning perl I am more concerned about creating something that works reasonably well rather than fully efficient at this point (though in major blunders would be welcomed if pointed out). So is there something I can do to stop these warnings? And is there something I am doing/not doing that would normally be a good way to have set up my arrays to have kept this from being a problem?

If you need more information or clarification just let me know as I will be checking back on this post often

EDIT: This is the line that the warning is coming from:

$RECORD[$det_line] = $sp x 4 .ltrim(substr($_, 0, 56).$sp x 26 .substr($_, 57, 14));

----Main Code----

#!/usr/bin/perl use strict; use warnings; use Path::Class; use autodie; use Text::CSV; #-----VARIABLES----- my @OUTPUT; my $sp = " "; my $seq_num = 0; my ($prev_amt_due, $payment_rec) = 0; my ($return_1, $return_2, $return_3, $return_4); my ($mail_to_1, $mail_to_2, $mail_to_3, $mail_to_4); my ($service_1, $service_2, $service_3, $service_4); my ($due_date, $bill_date, $amt_due, $t_amt_due, $past_amt, $cur_amt, +$phone); my ($cur_temp, $prev_temp, $ly_temp); my ($cur_days, $prev_days, $ly_days); my ($cur_usage, $prev_usage, $ly_usage); #-----MAIN----- my $dir = dir("C:/perl"); #Set the working directory my $file = $dir->file("c2_sample_file.txt"); #Set the working file open my $fh, '<:encoding(UTF-8)', $file; #Open the input file local $/ = "\014"; #Set Input Record Separator to line break #Loop through reading on record at a time while (<$fh>) { chomp; #Removes the control character my @LINES = split(/\n/, $_, -1); #Split on line break creating + array my @RECORDS = formatC2(@LINES); $OUTPUT[$seq_num] = [@RECORDS]; } close $fh; #Currently for checking data conversion accuracy for my $i ( 1 .. $#OUTPUT ) { for my $j ( 0 .. $#{ $OUTPUT[$i] } ) { print "elt $i $j is $OUTPUT[$i][$j]\n"; sleep 1; } } #-----SUBROUTINES----- sub ltrim { my $s = shift; $s =~ s/^\s+//; return $s }; sub rtrim { my $s = shift; $s =~ s/\s+$//; return $s }; sub trim { my $s = shift; $s =~ s/^\s+|\s+$//g; return $s }; sub writeCSV {} sub formatC2 { $seq_num += 1; #Increment for seq_num my @RECORD = @_; #Dereference the array my $needs_bre = 1; #Default to True for each record my $is_budget = 0; #Default to False for each record #Misc + Account Number $RECORD[0] = trim($RECORD[0]); #Trim line 1 -> L1 $RECORD[2] = trim($RECORD[2]); #Trim line 3 -> L3 $RECORD[5] = trim($RECORD[5]); #Trim line 6 -> Account Number #Return Address Block $return_1 = trim(substr $RECORD[7], 0, 38); $return_2 = trim(substr $RECORD[8], 0, 38); $return_3 = trim(substr $RECORD[9], 0, 38); $return_4 = trim(substr $RECORD[10], 0, 38); #Service Address Block $service_1 = trim(substr $RECORD[23], 0, 36); $service_2 = trim(substr $RECORD[24], 0, 36); $service_3 = trim(substr $RECORD[25], 0, 36); $service_4 = trim(substr $RECORD[26], 0, 36); #------------------------------------------------------------- #Conditions used to protect against short lines due to no data #-------------------------------------------------------------- #Mail To Address Block if (length($RECORD[7])>38) {$mail_to_1 = trim(substr $RECORD[7], +38, length($RECORD[7])-38); } if (length($RECORD[8])>38) {$mail_to_2 = trim(substr $RECORD[8], +38, length($RECORD[8])-38); } if (length($RECORD[9])>38) {$mail_to_3 = trim(substr $RECORD[9], +38, length($RECORD[9])-38); } if (length($RECORD[10])>38) {$mail_to_4 = trim(substr $RECORD[10], + 38, length($RECORD[10])-38); } #Bill and Due Date if (length($RECORD[23])>59) { $due_date = trim(substr $RECORD[23], + 58, 10); } if (length($RECORD[24])>59) { $bill_date = trim(substr $RECORD[24] +, 58, 10); } #Amount Due if (length($RECORD[25])>59) { $amt_due = trim(substr $RECORD[25], +58, 10); } #Total Amount Due if (length($RECORD[19])>56) { $t_amt_due = trim(substr $RECORD[19] +, 55, 15); } #Phone Number if (length($RECORD[55]) > 21) { $phone = trim(substr $RECORD[55], +21, 14); } #Temperature - Current, Previous, Last Year if (length($RECORD[55]) > 49) { $cur_temp = trim(substr $RECORD[55 +], 49, 5); } if (length($RECORD[56]) > 49) { $prev_temp = trim(substr $RECORD[5 +6], 49, 5); } if (length($RECORD[57]) > 49) { $ly_temp = trim(substr $RECORD[57] +, 49, 5); } #Days - Current, Previous, Last Year if (length($RECORD[55]) > 56) { $cur_days = trim(substr $RECORD[55 +], 56, 2); } if (length($RECORD[56]) > 56) { $prev_days = trim(substr $RECORD[5 +6], 56, 2); } if (length($RECORD[57]) > 56) { $ly_days = trim(substr $RECORD[57] +, 56, 2); } #Usage - Current, Previous, Last Year if (length($RECORD[55]) > 64) { $cur_usage = trim(substr $RECORD[5 +5], 64, 6); } if (length($RECORD[56]) > 64) { $prev_usage = trim(substr $RECORD[ +56], 64, 6); } if (length($RECORD[57]) > 64) { $ly_usage = trim(substr $RECORD[57 +], 64, 6); } #Detail Lines my $det_line = 29; #Set to first detail line while ($det_line <= 54) { local $_ = $RECORD[$det_line]; my $chkr = "This bill will be automatically drafted from your +bank account."; if(trim($_) eq $chkr) { $needs_bre = 0; } #No BRE if auto draf +t #If pattern match is true set $past_amt to "X" if (m/\s+.+\sbudget\s/) { $past_amt = "X"; $is_budget = 1; #Change message if both pattern and needs_bre conditi +ons are met if ($needs_bre == 0) { my $t_budg_amt = substr($RECORD[$det_line], 39, 1 +5); #Extract budget amount for new message $RECORD[$det_line-3] = " Your budget amount of + $t_budg_amt will be automatically drafted"; $RECORD[$det_line-2] = " from your bank accoun +t"; $RECORD[$det_line-1] = ""; $RECORD[$det_line] = ""; $det_line +=1; next; } } if (length($_) > 34){$chkr = substr($_, 34, 1);} my $chkr2 = substr($_, 0, 32); #Use Regex to check for pattern if ($chkr eq ".") { #Pad with additional spaces $RECORD[$det_line] = $sp x 4 .substr($_, 1, 4).$sp x +5 .substr($_, 7, 5).$sp x 8 .substr($_, 14, 5).$sp x 7 . + substr($_, 22, 1).$sp x 4 .substr($_, 26, 10).$sp x 9 .substr($_ +, 36,10).$sp.substr($_, 47, 10).$sp x 2 . substr($_, 57, 11); $det_line +=1; next; } elsif (m/^\sMeter\s#.+\.\d\d$/) { #Pad with additional spaces $RECORD[$det_line] = $sp x 4 .substr($_, 1, 18).$sp x + 53 .substr($_, 47, 10).substr($_, 57, 11); $det_line +=1; next; } elsif ($chkr2 =~ /^\s*$/){ if (m/^\s+Current\sCharges\s+\d+\.\d+$/) { $cur_amt += trim(substr $RECORD[$det_line], 56, 12); } if (m/^\s+Previous\sAmount\sDue\s+\d+\.\d+$/){ $prev_amt_due = trim(substr $RECORD[$det_line +], 56,12); $prev_amt_due =~ s/,//g; } if (m/^\s+Payment\sReceived\s/){ $payment_rec = trim(substr $RECORD[$det_line] +, 56,12); $payment_rec =~ s/,//g; } $RECORD[$det_line] = $sp x 26 .$RECORD[$det_line]; } else { $RECORD[$det_line] = $sp x 4 .ltrim(substr($_, 0, 56). +$sp x 26 .substr($_, 57, 14)); } if ($det_line == 54){ my $result = $prev_amt_due - $payment_rec; #See if la +st payment covered amt due if ($result > 0 and $is_budget == 0) { $past_amt = spr +intf("%.2f", $result); } } $det_line += 1; } #Create Return Array $det_line = 29; my @TEMP = sprintf "%05d", $seq_num; #Pad the seq_num push @TEMP, sprintf("%05d", $seq_num)."1".$needs_bre."1"; #Create +Barcode field push @TEMP, $RECORD[0], $RECORD[2], $RECORD[5]; push @TEMP, $return_1, $return_2, $return_3, $return_4; push @TEMP, $mail_to_1, $mail_to_2, $mail_to_3, $mail_to_4; push @TEMP, $service_1, $service_2, $service_3, $service_4; push @TEMP, $due_date, $bill_date, $amt_due, $t_amt_due, $phone; push @TEMP, $cur_temp, $prev_temp, $ly_temp; push @TEMP, $cur_days, $prev_days, $ly_days; push @TEMP, $cur_usage, $prev_usage, $ly_usage; push @TEMP, $cur_amt, $past_amt; while ($det_line <= 54) { push @TEMP, $RECORD[$det_line]; $det_line += 1; } @TEMP #Pass back array }

Replies are listed 'Best First'.
Re: Undef and Uninitialized Issue
by Athanasius (Cardinal) on Apr 02, 2015 at 14:03 UTC

    Hello Pharazon,

    It would help if you provided a sample of the input file, together with the output you are getting. At the least, you should specify the exact form of the warning message.

    In the meanwhile, I’m guessing that the problem arises in this expression: substr($_, 57, 14) when the string in $_ is less than 57 characters long. If so, you can remove the warning by adding a test along these lines:

    my $temp = substr($_, 0, 56) . $sp x 26; $temp .= substr($_, 57, 14) if length >= 57; $RECORD[$det_line] = $sp x 4 . ltrim($temp);

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

      I can't believe I missed that. I had conditioned so many of the other substr's to prevent issues and just complete missed that one. However, after clearing that out I am still getting on uninitialized error. Below is the sample input data and the output up to the error:

      INPUT

      06 01720168-00000000257980 123 N PLAZA HWY 123 172016-8 SOME COMPANY INC. ANOTHER COMPNAY OF SOMWHERE P O BOX 1234 123 STREET RD. CITY KY 12345 SUITE# 1234 CITY, ST 12345 6/23/2014 $257.98 ANOTHER COMPNAY OF SOMWHERE 6/23/2014 123 STREET RD. 172016-8 6/09/2014 CITY, ST 12345 $257.98 02CS 4/30 5/28 3117.0 3259.0 142.0 Meter # C204508 142.0 232.99 Pipe Replacement Pgm SNR Comm 3.27 RESEARCH & DEVELOPMENT TARIFF .03 3.00% Rate Increase County Co Sc Tax on 236.29 7.09 6.00% State Tax on 243.38 14.60 Current Charges 257.98 Previous Amount Due 351.60 Payment Received 5/22 351.60CR Total Amount Due 257.98 1-877-123-1234 66.0 28 142.0 8:00am to 4:00pm 58.6 30 203.0 70.3 28 174.0

      OUTPUT

      elt 1 0 is 00001 elt 1 1 is 00001111 elt 1 2 is 06 elt 1 3 is 01720168-00000000257980 elt 1 4 is 172016-8 elt 1 5 is SOME COMPANY INC. elt 1 6 is 123 STREET RD. elt 1 7 is CITY, ST 12345 elt 1 8 is elt 1 9 is ANOTHER COMPNAY OF SOMWHERE elt 1 10 is 123 STREET RD. elt 1 11 is SUITE# 1234 elt 1 12 is CITY, ST 12345 elt 1 13 is ANOTHER COMPNAY OF SOMWHERE elt 1 14 is 123 STREET RD. elt 1 15 is CITY, ST 12345 elt 1 16 is elt 1 17 is 6/23/2014 elt 1 18 is 6/09/2014 elt 1 19 is $257.98 elt 1 20 is $257.98 elt 1 21 is 1-877-123-1234 elt 1 22 is c66.0 elt 1 23 is 58.6 elt 1 24 is 70.3 elt 1 25 is 28 elt 1 26 is 30 elt 1 27 is 28 elt 1 28 is c142.0 elt 1 29 is 203.0 elt 1 30 is 174.0 elt 1 31 is 257.98 Use of uninitialized value in concatenation (.) or string at c:/perl/c +onvert2.pl line 45. elt 1 32 is elt

        That particular warning comes about because this statement:

        push @TEMP, $cur_amt, $past_amt;

        towards the end of sub formatC2 is called with $past_amt undefined. You can fix this by providing $past_amt with a default value near the start of the subroutine:

        $past_amt = '';

        I tracked this down by looking at the contents of @OUTPUT after the first while loop finishes. As fishmonger says, you can use Data::Dumper or Data::Dump for this. I prefer Data::Dump:

        use Data::Dump; ... while (<$fh>) { chomp; #Removes the control character my @LINES = split(/\n/, $_, -1); #Split on line break creating + array my @RECORDS = formatC2(@LINES); $OUTPUT[$seq_num] = [@RECORDS]; last; } close $fh; dd \@OUTPUT; exit(0);

        The last ensures that I get only the first iteration of the while loop, and the exit helps me to see what I need to see (in this case, the contents of @OUTPUT) by omitting everything that comes after.

        Hope that helps,

        Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

Re: Undef and Unitialized Issue
by fishmonger (Chaplain) on Apr 02, 2015 at 14:09 UTC

    There are a number of issues with your code, but I'll focus on your uninitialized warnings question.

    Have you tried adding any debugging print statements to help track down the problem?

    Use the Data::Dump or Data::Dumper module to dump each of the vars in the statement that is generating the warning to verify they contain the values you expect. For each var that is undefined/uninitialized, trace it back step-by-step until you find where it began to be undefined. Adjust the code at that point to handle it, which could mean to assign a default value or it could mean that you should move to the next iteration.

      I mentioned in my post that I felt I had it tracked to a particular area of my code, however Athanasius pointed out a pretty obvious issue with that particular line which did resolve part of the problem. You can see my response to them for further details. I am however curious what issues there are with my code

      I'm not a perl programmer although having started working with the language I quite like it and will likely put some time into it going forward. It's just that the software packages we use in house are not capable of dealing with a file like this and so I was asked to come up with a solution and this is me trying to do that. Any advice is appreciated though if its very technical in relation to perl specific things it will likely be beyond me for the time being though I would still like to know.

      I know I can use the Datta::Dump to see what is going on but I have a loop that does something similar showing me the position in the 2d array as well since I was double checking positioning for the eventual output to csv format.

Re: Undef and Unitialized Issue
by boftx (Deacon) on Apr 02, 2015 at 20:11 UTC

    In more general terms, I typically use something like this to silence those warnings (assuming a value of undef is not an error):

    my $string_var = defined( $input_field ) || ''; # or if ( ($string_var || '') !~ /some_pattern/ ) { ... }
    If in fact undef is an error I would generally use something along this line:
    croak "HEY! Why am I not defined?" if !defined( $input_field );

    You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.
      my $string_var = defined( $input_field ) || '';

      This will set $string_var either to 1 or to "", no matter what $input_field was set to. Looking at the variable names, this is hardly what you want.

      ($string_var || '')

      This evaluates to "" if $string_var is undef, 0, or "0". You don't want to change "0" or 0 to the empty string, do you?

      Why don't you use the defined-or operator //? $string_var//'' evaluates to "" only if $string_var is "" or undef. 0 and "0" are not changed.

      Ancient perl versions have no defined-or-operator, so you have to resort to something like $never_undef=defined($perhaps_undef) ? $perhaps_undef : ''; for code that is expected to work with ancient perl verisons.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

        You are correct, I was thinking of where I use delete($input) || '' to ensure a defined value and "0" is not a concern. The second form you present (using ?:) is more correct in the general case.

        As for using an "ancient version of Perl" I don't have a choice at $work, we are stuck on 5.8.9 for at least a few more months.

        You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.