gaseous1 has asked for the wisdom of the Perl Monks concerning the following question:

I am printing in a WHILE loop, using a text database file that is tab delimited. There is a header line, followed by several thousand data lines. Each record contains 21 fields.

When I print in the WHILE loop, very strange things happen. I am printing to STDOUT and to an output file. The STDOUT is currently used for debugging, but will eventually be used as a progress indicator. The output file will be used to store summarized data from the original file.

The madness happens with the print statements. The behavior is similar regardless of printing to STDOUT or to a file. The first line which the program evaluates in the input file, everything prints as expected. On subsequent iterations, the **only** item that prints is the $line value (actually, I think it is the $_ value). Nothing else prints.

I've looked far and wide for an answer and can't find one. Why is this happening? How do I rectify it?

#!/usr/bin/perl
use strict;
use warnings;

my $in_fileName = "/Users/me/test.txt";
my $out_fileName = "/Users/me/test-out.txt";

my $infi;
my $oufi;
my @data_array;
my $line;
my $ctr;
my $txt1 = " Field:\t";

open $infi, "<" . $in_fileName or die $!;
open $oufi, ">" . $out_fileName or die $!;

while(<$infi>)
{
    $line = $_;
    chomp($line);
    @data_array = split(/\t/, $line);
    for ($ctr = length[@data_array]; $ctr >= 0; $ctr--) {
        print {*STDOUT} $txt1, $data_array[$ctr], "\t", $ctr + 1, "\t"
+, $., "\n";
    }
    print {$oufi} "Record: ", $line,"\n";
}
close $oufi;
close $infi;

[download]

Replies are listed 'Best First'.
Re: Printing in a WHILE loop
by Athanasius (Archbishop) on May 27, 2014 at 04:19 UTC

    Hello gaseous1, and welcome to the Monastery!

    Consider:

    14:08 >perl -wE "my @a = (1, 3, 7); say @a; say [@a]; say length[@a];"
137
ARRAY(0x53b0d0)
15

14:11 >

    [download]

    The length here is 15 because there are 15 characters in “ARRAY(0x53b0d0)”, the reference string seen by the length function (which evaluates its operand in scalar context). Probably, you meant:

    for ($ctr = $#data_array; $ctr >= 0; $ctr--) {

    [download]

    although this would be better written as:

    for (reverse 0 .. $#data_array) {

    [download]

    Update: Just to clarify: In the above code I use $#data_array rather than scalar @data_array because the former gives the highest array index, whereas the latter gives the number of elements in the array. Since Perl arrays are indexed starting at zero, attempting to access element $data_array[scalar @data_array] is usually a logic error; for example, if the array has 3 elements, $data_array[3] accesses an uninitialised element:

    14:49 >perl -wE "my @x = (12, 34, 567); say $x[3];"
Use of uninitialized value in say at -e line 1.


14:50 >

    [download]

    Hope that helps,

    Athanasius <°(((>< contra mundum Iustus alius egestas vitae, eros Piratica,

[reply]
[d/l]
[select]
      Thanks for your help and clarification. I am still getting used to Perl's syntax (which sometimes is rather obscure to the uninitiated).
[reply]
Re: Printing in a WHILE loop
by NetWallah (Canon) on May 27, 2014 at 04:20 UTC
    Your biggest (bug causing) problem is the use of: 
     length[@data_array]

    [download]
    instead of 
     length (@data_array)

    [download]
    There are other style-related issues, primarily dealing with localizing the scope of declared variables, and using a "c-style" "for" loop where a "perl style" would be more appropriate. Let me know if you need elaboration on either point.

    UPDATE: length (@data_array) will not give you expected results either . (Thanks choroba for pointing that out) ; use 

    scalar(@data_array)

    [download]
    instead.

    This is because length() takes a scalar. These statements are equivalent (return the same value): 

          length (@data_array) ==  length (scalar @data_array)

    [download]
    So the value returned by length is the length in bytes of the size of the array.
    I.e. if there are 900 elements in the array, length() would return "3" (Whereas scalar() would return 900).

            What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?
                  -Larry Wall, 1992

[reply]
[d/l]
[select]
      length @array really? It might be useful in some other languages, but in Perl, length is not scalar.
      لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
[reply]
[d/l]
        'Tis difficult to get rid of old habits, learned many years ago. Perl is something I go back to from time to time, and therefore is always relatively "new" to me.
[reply]
      Thanks for your comments, as they have been most helpful. I'd happily take any further suggestions you have.
[reply]
Re: Printing in a WHILE loop
by GrandFather (Saint) on May 27, 2014 at 12:00 UTC

    There are a bunch of things there that can be improved to make your life easier. The first thing is that the "text database" you are using is a CSV file and they are very common so of course there is pre-written code just waiting to make life easier for you. So let's see how that looks:

    use strict;
use warnings;
use Text::CSV;

my $fileStr = <<FILE;
Line,Name,Colour,Comment
1,Fred,Blue,"Comment with a comma, and stuff."
2,Joe,Green,No comma so no need for quotes
3,Bob,Red,"""A comma, and quotes"""
4,Sam,Yellow,"Penultimate line, but
split across two input lines"
5,Bill,Violet,And an ordinary line to finish
FILE

my $csv = Text::CSV->new({binary => 1, eol => $/});

open my $io, "<", \$fileStr;
my @headers = @{$csv->getline($io)};

while (my $row = $csv->getline($io)) {
    print "File line $.\n";
    print "  $headers[$_]: '$row->[$_]'\n" for reverse 0 .. $#$row;
}

    [download]

    Prints:

    File line 2
  Comment: 'Comment with a comma, and stuff.'
  Colour: 'Blue'
  Name: 'Fred'
  Line: '1'
File line 3
  Comment: 'No comma so no need for quotes'
  Colour: 'Green'
  Name: 'Joe'
  Line: '2'
File line 4
  Comment: '"A comma, and quotes"'
  Colour: 'Red'
  Name: 'Bob'
  Line: '3'
File line 6
  Comment: 'Penultimate line, but
split across two input lines'
  Colour: 'Yellow'
  Name: 'Sam'
  Line: '4'
File line 7
  Comment: 'And an ordinary line to finish'
  Colour: 'Violet'
  Name: 'Bill'
  Line: '5'

    [download]

    I used commas instead of tabs so that it's easier to see the separator characters. Note though that I included some fairly nasty stuff in the comment data: commas, quotes and even a line break. Text::CSV just gobbled the whole lot up and asked for more. How would your split have handled that data?

    To make this a stand alone sample I used a string as a file.

    Note the for reverse 0 .. $#$row loop across the @row and @header arrays. Much clearer and easier to get correct than a C style array. Oh, and for used after the print statement like that it's a "statement modifier" - a neat trick for one line loops over short statements. You can use while and if in similar fashion.

    Perl is the programming world's equivalent of English
[reply]
[d/l]
[select]
      The data source is tab delimited and of consistently formatted. I would be happy to install a library, but I am fairly new to Perl and am more interested in learning the language. I am also aware of the limitations of "split" but it serves my purposes for this project. I appreciate your comments, and would be happy to hear whatever else you wish to comment on.
[reply]
      Very much like the reverse loop and your description. Will use this construct in the future.
[reply]
Re: Printing in a WHILE loop
by wjw (Priest) on May 27, 2014 at 04:54 UTC

    Code:

    #!/usr/bin/perl
use strict;
use warnings;

#my $in_fileName = "/Users/me/test.txt";
#my $out_fileName = "/Users/me/test-out.txt";
my ($infi, $oufi, @data_array, $line, $ctr);
my $txt1 = " Field:\t";
#open $infi, "<" . $in_fileName or die $!;
#open $oufi, ">" . $out_fileName or die $!;
while(<DATA>) {
    $line = $_;
    chomp($line);
    @data_array = split(/\t/, $line);
    $ctr = (@data_array) - 1;
    for ($ctr; $ctr >= 0; $ctr--) {
        print "$txt1,$data_array[$ctr],\t$ctr + 1, \t$.\n";
    }
    print  "Record: ", $line,"\n";
}
#close $oufi;
#close $infi;


__DATA__
1    2    3    4    5    6    7    8    9    10    11    12    13    1
+4    15
1a    2a    3a    4a    5a    6a    7a    8a    9a    10a    11a    12
+a    13a    14a    15a

    [download]
    Outupt:
     Field:    ,15,    14 + 1,     1
 Field:    ,14,    13 + 1,     1
 Field:    ,13,    12 + 1,     1
 Field:    ,12,    11 + 1,     1
 Field:    ,11,    10 + 1,     1
 Field:    ,10,    9 + 1,     1
 Field:    ,9,    8 + 1,     1
 Field:    ,8,    7 + 1,     1
 Field:    ,7,    6 + 1,     1
 Field:    ,6,    5 + 1,     1
 Field:    ,5,    4 + 1,     1
 Field:    ,4,    3 + 1,     1
 Field:    ,3,    2 + 1,     1
 Field:    ,2,    1 + 1,     1
 Field:    ,1,    0 + 1,     1
Record: 1    2    3    4    5    6    7    8    9    10    11    12   
+ 13    14    15
 Field:    ,15a,    14 + 1,     2
 Field:    ,14a,    13 + 1,     2
 Field:    ,13a,    12 + 1,     2
 Field:    ,12a,    11 + 1,     2
 Field:    ,11a,    10 + 1,     2
 Field:    ,10a,    9 + 1,     2
 Field:    ,9a,    8 + 1,     2
 Field:    ,8a,    7 + 1,     2
 Field:    ,7a,    6 + 1,     2
 Field:    ,6a,    5 + 1,     2
 Field:    ,5a,    4 + 1,     2
 Field:    ,4a,    3 + 1,     2
 Field:    ,3a,    2 + 1,     2
 Field:    ,2a,    1 + 1,     2
 Field:    ,1a,    0 + 1,     2
Record: 1a    2a    3a    4a    5a    6a    7a    8a    9a    10a    1
+1a    12a    13a    14a    15a

    [download]

    Hope that is helpful...

    ...the majority is always wrong, and always the last to know about it...

    Insanity: Doing the same thing over and over again and expecting different results...

    A solution is nothing more than a clearly stated problem...otherwise, the problem is not a problem, it is a facct

    wjw
[reply]
[d/l]
[select]
      Thank you very much! Excellent.
[reply]
Re: Printing in a WHILE loop
by GrandFather (Saint) on May 28, 2014 at 07:04 UTC

    Or, since you used the keyword "database", how about:

    use strict;
use warnings;
use DBI;

open my $fOut, '>', "delme" or die "Can't create deleme.txt: $!\n";
print $fOut <<FILE;
Line,Name,Colour,Comment
1,Fred,Blue,"Comment with a comma, and stuff."
2,Joe,Green,No comma so no need for quotes
3,Bob,Red,"""A comma, and quotes"""
4,Sam,Yellow,"Penultimate line, but
split across two input lines"
5,Bill,Violet,And an ordinary line to finish
FILE
close $fOut;

my $dbh = DBI->connect('dbi:CSV:', '', '', {f_file => 'delme'});
my $sth = $dbh->prepare("select * from delme");

$sth->execute();

while (my $row = $sth->fetchrow_hashref()) {
    print "$_: '$row->{$_}'\n" for sort keys %$row;
    print "\n";
}

    [download]

    Prints:

    colour: 'Blue'
comment: 'Comment with a comma, and stuff.'
line: '1'
name: 'Fred'

colour: 'Green'
comment: 'No comma so no need for quotes'
line: '2'
name: 'Joe'

colour: 'Red'
comment: '"A comma, and quotes"'
line: '3'
name: 'Bob'

colour: 'Yellow'
comment: 'Penultimate line, but

split across two input lines'
line: '4'
name: 'Sam'

colour: 'Violet'
comment: 'And an ordinary line to finish'
line: '5'
name: 'Bill'

    [download]

    No serious intent that you should use this for your current problem, but it does go some way to show just how flexible DBI is!

    Perl is the programming world's equivalent of English
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom