orthanc has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks

Can anyone explain why the code below destroys the contents of @test_array?

All I can see is that we are simply reading from @test_array and never editing it, if the while statement in the subroutine is changed to a

foreach $line (<IN>)

@test_array remains unaffected.

I'm baffled, should the $_ variable being used for the file not be in a different scope to @test_array?

The size of @test_array remains unaffected but each element is empty, thus the warnings for uninitialised variables.

please say its not something blatantly obvious!!!
Orthanc

#!/usr/local/bin/perl -w my @test_array=( 'Element1', 'Element2', 'Element3', 'Element4', 'Element5', ); foreach (@test_array) { print "\"$_\"\n"; do_something(); } print "\n"; foreach (@test_array) { print "\"$_\"\n"; } sub do_something { open(IN, "$0") || die("No file"); while (<IN>) { # Do nothing, in the real software we would do something # with the content. } close(IN); }

Replies are listed 'Best First'.
Re: What Is Going On Here ?????
by autark (Friar) on Nov 21, 2000 at 22:03 UTC
    There is one important things to note about foreach and while loops, namely the localization of the loop variable.

    The foreach loop will always localize its loop variable (That is $_ unless one is spesified). This can easily be verified:

    $_ = 42; foreach (1, 2, 3) { print "Inside loop: \$_ = [$_]\n"; } print "Outside loop: \$_ = [$_]\n";
    This will print
    Inside loop: $_ = [1]
    Inside loop: $_ = [2]
    Inside loop: $_ = [3]
    Outside loop: $_ = [42]
    
    The while loop however, does not localize anything. This can be seen from the following example
    $_ = 42; while(<FH>) { print "Inside loop: \$_ = [$_]"; } print "Outside loop: \$_ = [$_]\n";
    If <FH> produces three lines containing the digits 1, 2 and 3 the output will be
    Inside loop: $_ = [1]
    Inside loop: $_ = [2]
    Inside loop: $_ = [3]
    Outside loop: $_ = []
    
    It will also produce a warning if you remembered to turn on the -w flag. Why the warning you might ask. The reason is because the <FH> will return undef when it has reached the end of the file, and $_ is assigned this undef. And as we all know, perl will warn us if we use an undefined value (if the -w flag is used).

    So then, what is it that happens in your program ? First, let's just simplify it a little bit (concentrating on the first foreach loop):

    foreach (@test_array) { print qq|Before while: [$_]\n|; open(IN, $0) || die "No file: $!\n"; while (<IN>) { } close IN; print qq|After while: [$_]\n|; }
    Now, what happens is that the foreach loop creates spesial references into the @test_array. (Spesial in the sense that when you change $_, you also change the corresponding element in the @test_array)

    The next line should then print out the value of $_, which in your example would be "Element1". Then we open ourself, and loop over each line in the file. The while (<IN>) construct implicitply assigns the $_ without first localizing it. Thus we actually change the elements of the array @test_array.

    If you instead had used a foreach loop to iterate over the file, this problem would not have occured, because of the fact that it would have localized the $_ variable for you. Or you could localize $_ yourself by writing local $_. Another possible solution is to use lexical variables both in the foreach loop and in the while loop

    foreach my $elem (@test_array) { open(IN, $0) || die "No file: $!\n"; while( defined(my $line = <IN>) ) { } close IN; }
    (The defined test isn't really necessary in newer perls, but I like to be explicit here).

    "But", I hear you complain, "my while loop is in a different function, a completly new scope". Well, that is correct if you are talking about lexical scoping (variables declared with the my keyword are lexical variables). However, $_ is a dynamically scoped variable (this fact you can't change, you can't declare my $_. Here is a good tutorial explaining the difference between a lexical scope and a dynamic scope.) However, since $_ is dynamically scoped, it will live on in the new function only to be assigned by the while loop.

    I hope this shed some light on the problem at hand.

    Autark.

Re: What Is Going On Here ?????
by Jonathan (Curate) on Nov 21, 2000 at 15:42 UTC
    When you loop over your array $_ becomes an alias for each item so you are in fact modifying the array.
    From perlfaq4

    How do I process/modify each element of an array? Use for/foreach: for (@lines) { s/foo/bar/; # change that word y/XZ/ZX/; # swap those letters }


    And now, as he looked and saw the whole Hellespont covered with the vessels of his fleet, and all the shore and every plain about Abydos as full as possible of men, Xerxes congratulated himself on his good fortune; but after a little while he wept...
    Asked why he was weeping he replied
    ..."There came upon me," replied he, "a sudden pity, when I thought of the shortness of man's life, and considered that of all this host, so numerous as it is, not one will be alive when a hundred years are gone by."
    Recorded by Herodotus Written about 440 B.C.

      I am aware of what you are saying, but my understanding is that the $_ variable of the test_array foreach loop would be in a different scope to the $_ variable in the do_something subroutine.

      Orthanc

        $_ is a global variable. It is best to save it for inner loops, but if you must do it like this give it a seperate value for the scope of do_something with local($_).

        Updated: Okay thanks for correcting me extremly, but it seem to me to make more sense (and is clearer) if $_ is saved for innermost loops.
        For example this could be clearer:
        while (<>) { for (split /-/,$_) { print $_ . "\n"; } }
        As repson says $_ is a global, to preserve your initial array I guess you'd have to protect it with
        while (local $_ = <IN>) {
        in the do_something sub

        And now, as he looked and saw the whole Hellespont covered with the vessels of his fleet, and all the shore and every plain about Abydos as full as possible of men, Xerxes congratulated himself on his good fortune; but after a little while he wept...
        Asked why he was weeping he replied
        ..."There came upon me," replied he, "a sudden pity, when I thought of the shortness of man's life, and considered that of all this host, so numerous as it is, not one will be alive when a hundred years are gone by."
        The History of Herodotus By Herodotus Written about 440 B.C.
Re: What Is Going On Here ?????
by Dominus (Parson) on Nov 21, 2000 at 22:23 UTC
    This is why subroutines that use $_ should always begin with local $_;:
    sub do_something { local $_; ... }
    will fix the problem.

      I just wanted to emphasize that this is a really important point that will save you from the kinds of bugs that can take forever to track down.

      With modern Perls, you can skip the local $_ if you only ever use $_ as a loop variable, but better to be safe if you aren't sure.

      Update: Note that this doesn't apply to while loops (which don't have a loop variable in my book) as nicely pointed out by autark in Re: What Is Going On Here ?????.

              - tye (but my friends call me "Tye")
Re: What Is Going On Here ?????
by dws (Chancellor) on Nov 21, 2000 at 22:57 UTC
    The other approach is to avoid letting foreach default to using $_, and avoid passing $_ as an implicit argument (which your code doesn't seem to do, though I suspect you may have meant to open $_ instead of $0).
        foreach my $element ( @array ) {
            print "\"$element\"\n";
            do_something($element);
        }
    
    An explicit loop variable helps express the code's intention, which helps when some poor fool has to pick up the code six months later.

      Just like to thank everyone for their insights into this.

      and a p.s. to dws, i did mean $0 for the filename, so the script would just open itself when it was run, it saves having to supply a test file :)

      Thanks All,
      Orthanc

Re: What Is Going On Here ?????
by Anonymous Monk on Nov 23, 2000 at 09:30 UTC
    Hi,

    Variable $_ is a global variable and loops like while does not preserve original value of $_. If you use foreach loop inside while loop, that will be fine, since foreach loop preserves original value of $_, whenever loop ends, but while doesn't. So, you should use another variable, for simplicity, and continue ...

    #! /usr/bin/perl -w my @test_array=( 'Element2', 'Element3', 'Element4', 'Element5', ); my $item; foreach $item (@test_array) { print "\"$item\"\n"; &do_something(); } print "\n"; foreach $item (@test_array) { print "\"$item\"\n"; } sub do_something { open(IN, "$0") || die("No file"); while (<IN>) { # Do nothing, in the real software we would do something # with the content. } close(IN); }