dbanas has asked for the wisdom of the Perl Monks concerning the following question:

Why is the $_ variable of the outer foreach() loop getting clobbered by the inner while() loop in the code, below?
1 #! /usr/bin/perl -w 2 3 my @test_files = glob "*.txt"; 4 5 foreach (@test_files) { 6 print "Starting processing of file, $_.\n"; 7 open TEST_FILE, "< $_" or die "Couldn't open test file, '${_ +}', for reading: ($!)"; 8 9 while (<TEST_FILE>) { 10 print; 11 } 12 13 print "Ending processing of file, $_.\n"; 14 }
When I run the code, above, I get this output: (The local directory contains 2 *.txt files: test1.txt and test2.txt.)
Starting processing of file, test1.txt. Line 1 from 'test1.txt' Line 2 from 'test1.txt' Line 3 from 'test1.txt' Use of uninitialized value in concatenation (.) or string at ./test.pl + line 13, <TEST_FILE> line 3. Ending processing of file, . Starting processing of file, test2.txt. Line 1 from 'test2.txt' Line 2 from 'test2.txt' Line 3 from 'test2.txt' Use of uninitialized value in concatenation (.) or string at ./test.pl + line 13, <TEST_FILE> line 6. Ending processing of file, .
Note the missing filenames at the ends of both "Ending processing..." lines, as well as the warnings immediately preceeding them. Here are the contents of "test1.txt" and "test2.txt":
test1.txt:
Line 1 from 'test1.txt' Line 2 from 'test1.txt' Line 3 from 'test1.txt'
test2.txt:
Line 1 from 'test2.txt' Line 2 from 'test2.txt' Line 3 from 'test2.txt'
Thanks!

Replies are listed 'Best First'.
Re: $_ getting clobbered by inner loop.
by pc88mxer (Vicar) on Apr 08, 2008 at 23:49 UTC
    Unfortunately, that's just the way it is. You are better off using a lexical variable for (at least) the outer loop:
    for my $file (@test_files) { # use $file here while (<TEST_FILE>) { # use $_ here } }
    The default loop variable $_ is only meant to be used for small code fragments. For anything more complicated you should use a named lexical.

    Update: Your example does illustrate the difference between for loops and while loops. In the following nested loops, $_ does not get clobbered:

    for (1..3) { print "_ before = $_\n"; for ('a'..'c') { print "_ inner loop = $_\n"; } print "_ after = $_\n"; # prints same as before inner loop }
Re: $_ getting clobbered by inner loop.
by bigmacbear (Monk) on Apr 08, 2008 at 23:57 UTC

    It's actually quite simple: the diamond operator (in your example, "<TEST_FILE>" is the diamond operator with the filehandle TEST_FILE as its argument) reads one line from the filehandle supplied and puts it in $_, clobbering whatever was there. A bare "print;" then uses the implied $_ variable as its argument.

    I see two options: either localize $_ by using "local $_;" in a scope outside your while loop but within your foreach loop, or use a lexical variable in your outer scope (e.g. "foreach my $file (@test_files)") and then substitute it for $_ in your outer loop. In my opinion the latter is much clearer and thus preferred.

    I could also have suggested a replacement for the automatic use of $_ in the inner loop, but the benefits you gain from using the implied $_ variable here are greater than those you lose by not using it in your outer loop.

Re: $_ getting clobbered by inner loop.
by ysth (Canon) on Apr 09, 2008 at 03:16 UTC
    while (<HANDLE>) is short for while (defined($_ = <HANDLE>)), which sets $_ but does not localize it. You can use a different variable explicitly: while (my $line = <HANDLE>) (the implied "defined" on the result of the assignment is still added), or, beginning in 5.10, you can lexicalize $_:
    foreach (@blah) { do stuff with $_ from @blah { my $_; while (<HANDLE>) { do stuff with $_ read from the handle } } do stuff with $_ from @blah }
Re: $_ getting clobbered by inner loop.
by GrandFather (Saint) on Apr 09, 2008 at 00:24 UTC

    Note too that the 3 argument version of open is safer and clearer than the two argument version in your sample code:

    open TEST_FILE, '<', $_ or die ...

    Perhaps also of interest is that the while print loop can be rewritten:

    print while <TEST_FILE>;

    or even:

    print <TEST_FILE>;

    although I presume you have provided sample code and such trivial stuff doesn't apply in your real application.

    For various reasons rewriting your for loop header as:

    for my $filename (@test_files) {

    is a good thing to do. The immediate reason is that it fixes your problem (as others have mentioned already). It also better documents your code and may facilitate debugging it. You do of course need to use $filename in place of $_ as appropriate.


    Perl is environmentally friendly - it saves trees
Re: $_ getting clobbered by inner loop.
by lestrrat (Deacon) on Apr 08, 2008 at 23:50 UTC
    while (<>) places each line to $_, and $_ is a global, so there. to avoid it (or actually, as a general rule), don't use $_ gratuitously. use an explicit variable.
Re: $_ getting clobbered by inner loop.
by ikegami (Patriarch) on Apr 09, 2008 at 00:48 UTC
    You can nest for loops because each for loop localizes its iterator. However, while doesn't localize $_ (or even know anything about it). You'll need to do it explicitly.
      Ah, I see. Thanks. Can you explain the philosophy behind this design choice? That is, does localizing $_ in for loops, but not in while loops, provide something useful to Perl? Thanks. -db

        Localizing $_ in a while loop would cause any changes to that variable to mysteriously be lost when you exit the while.

        Code should not mess around with things that are not its concern, since that would generate unintended and unexpected side effects. The while loop does not inherently use $_, so therefore it should not localize or otherwise muck up $_.

        The same reasoning would explain why a while loop does not automatically localize @test_files as well.

        Because $_ has nothing to do with while loops. You're looking at one specific while loop (while (defined($_ = <FILE>)) when asking that question, but while is used in many other ways too. For example, why should Perl localize $_ in while (@todo)? Or in while (my ($key, $val) = each %hash)?
        larry he is not