S_Shrum has asked for the wisdom of the Perl Monks concerning the following question:

What I am doing
==================================

I'm attempting to sum the values of a numeric column in a multidim array. When I use the following line of code, I get a null result:

my $columnsum += $_->[$colnum] for @data;

However, if I do the following instead, the summed value is correct:

$columnsum = 0; $columnsum += $_->[$colnum] for @data;

Why is the 'my' causing me such headaches???

TIA

======================
Sean Shrum
http://www.shrum.net

Replies are listed 'Best First'.
Re: 'my' headache...why doesn't this work?
by demerphq (Chancellor) on Apr 05, 2002 at 11:39 UTC
    If you do a search on CLPM you'll find this comes up semi-regularly, and I bet there are even a few threads here at the monastery too.

    Contrary to the many opinions stated here it is actually NOT localizing the variable. This should be obvious because strict doesnt get mad, and there is no curly brace to indicate a scoped block, and furthermore modifiers operate in the scope they are used and do not create a scope. Instead its reiniting it each time.

    What happens is that on the last call to the for (the one where the for has now exhausted the array or more generally evaluated as finished) the my part of the statement is STILL being executed, and thus causing the variable to go to undef.

    BTW: Sean in future try to be a bit more careful about the term "null", its meaning is overloaded in the CS world and can lead to confusion. Perl has no "NULL" but it does have undef and no they arent the same.

    HTH, and when you do your search youll see that I asked this question a long time ago on CLPM too. :-)

    Oh and the moral is that you shouldnt mix lexical declaration with modifiers.

    Yves / DeMerphq
    ---
    Writing a good benchmark isnt as easy as it might look.

Re: 'my' headache...why doesn't this work?
by ViceRaid (Chaplain) on Apr 05, 2002 at 10:12 UTC

    Hi

    Consider it written this way around instead

    foreach (@data) { my $columnsum += $_->[$colnum]; } print $columnsum; # is undef, and would give a strict error

    Your $columnsum is being repeatedly re-localised at each step of the foreach loop, and doesn't get a value outside of it.

    Interestingly, use strict doesn't seem to help here. I'd either expect strict to complain about a subsequent use of $columnsum outside of the (implicit) scope of the post-positional for loop, or for warnings to complain about masking a previously declared lexical variable. Can any wise folk here explain why it works like this?

    //=\\
Re: 'my' headache...why doesn't this work?
by mla (Beadle) on Apr 05, 2002 at 10:29 UTC
    The B::Deparse module really helps figure this sort of thing out.
    Here's the test code I wrote:

    #!/usr/bin/perl -w use strict; my @data = ([3], [8], [5]); my $sum += $_->[0] for @data; print "Total: $sum\n";

    And perl -MO=Deparse,p /tmp/try spits out:

    my(@data) = ([3], [8], [5]); foreach $_ (@data) { my $sum += $$_[0]; } print "Total: $sum\n";

    I agree though, weird that use strict doesn't complain.

      Update
      crazyinsomniac suggested that I expand on my comments here so this node has been changed from its original. (Oh and I retitled it too.)

      And as should always be kept in mind when using B::Deparse, the code output can be and in this case is clearly wrong.

      Obviously if it was correct the my clause would occur before the for.

      Ok, well apparently this isnt so obvious so here it is again. Statement Modifiers do not scope the statement that they modify. For instance the following are not the same because the scope they execute in is different.

      print "In main scope $_\n" for 1..10; for (1..10) { print "In scoped block:$_\n"; }
      So when B::Deparse takes the code of the first example and naively converts it to the second it has produced incorrect output. This can be seen by the following
      D:\Development>copy con deparse.pl use strict; my $x=$_ foreach 1..10; print $x; ^Z 1 file(s) copied. D:\Development>copy con deparse.out.pl use strict; ^Z 1 file(s) copied. D:\Development>perl -MO=Deparse deparse.pl >>deparse.out.pl deparse.pl syntax OK D:\Development>perl deparse.pl D:\Development>perl deparse.out.pl Global symbol "$x" requires explicit package name at deparse.out.pl li +ne 5. Execution of deparse.out.pl aborted due to compilation errors. D:\Development>
      So obviously as the input code compiles under strict but the output wont, the two cannot be the same.

      Its interesting actually that B::Deparse handles conditional and while loop modifiers correctly, but not for loop modifiers.

      D:\Development>perl -MO=Deparse -e "use strict; my $x=1; my $y=1 if $x +; print $x.' '.$y;" my $x = 1; my $y = 1 if $x; print $x . ' ' . $y; -e syntax OK D:\Development>perl -MO=Deparse -e "use strict; my $x=10; my $y=$x whi +le ($x--); print $x.' '.$y" my $x = 10; my $y = $_ while $x--; print $x . ' ' . $y; -e syntax OK
      Incidentally we can reduce this bug down to the following very simple example
      D:\Development>perl -MO=Deparse -e "use strict; my $x=1 for 1; print $ +x" foreach $_ (1) { my $x = 1; } print $x; -e syntax OK D:\Development>perl -e "use strict; my $x=1 for 1; print $x"
      This clearly shows what is going wrong here (note the output wont compile under strict) and it clearly shows that at least with regard to for modifiers B::Deparse produces incorrect output. (There are other examples listed in the B::Deparse documentation.)

      Always be careful when using B::Deparse to make comments about what Perl is doing. B::Deparse isnt perl and only makes a good (well, very good) attempt at the translation.

      Yves / DeMerphq
      ---
      Writing a good benchmark isnt as easy as it might look.

Re: 'my' headache...why doesn't this work?
by ls (Scribe) on Apr 05, 2002 at 10:06 UTC
    The left side of the for-Statement is executed multiple times, so with my, $columnsum seems to become a local variable in the for-block, although use strict doesn't complain.
    Try
    my $columnsum = 0; $columnsum += $_->{columns} for @data; print $columnsum;
    and everything seems to be fine.

      That's works....I guess I understand the FOR issue. Still would'a thunk that strict would have gagged on it.

      ======================
      Sean Shrum
      http://www.shrum.net

        strict is smart enough to detect variables that are not properly declared, but not necessarily to detect variables that are logically misused. In this case, declaring a 'my' variable inside a for loop is perfectly legal (in fact, I do it all the time without thinking much about it), e.g.
        foreach my $foo (@ary) { . . . }
        But for a running summation, of course it's not what you want. So strict doesn't see a problem; although it's misused, the variable is properly declared.
Re: 'my' headache...why doesn't this work?
by particle (Vicar) on Apr 05, 2002 at 13:40 UTC
    the only way i could think of you declaring and assinging on one line was like this, sean. as you can see, this code has been obfuscated into oblivion. don't do this.

    my @data = ( 1..4 ); $$_ = eval{ local $"="+"; eval "@data" } for \my $columnsum;
    i've seen this technique (for \my ...) used as a cute way to build a hash from two arrays

    @$_{@keys} = @values for \my %hash;
    but i prefer the more legible, highly maintainable

    my %hash; @hash{@keys} = @values;

    ~Particle ;Þ

Re: 'my' headache...why doesn't this work?
by S_Shrum (Pilgrim) on Apr 05, 2002 at 10:05 UTC

    For those wondering if I am losing the variable outside its 'my' scope: I am printing the result immediately after the calculations, like:

    my $columnsum += $_->[$colnum] for @data; print "Total value = $columnsum";

    ...and...

    $columnsum = 0; $columnsum += $_->[$colnum] for @data; print "Total value = $columnsum";

    ======================
    Sean Shrum
    http://www.shrum.net

      # your code my $columnsum += $_->[$colnum] for @data; # is effectively the same as for (@data) { my $columnsum += $_->[$colnum] } # so as expected $columnsum is effectively localised to that block (li +ne). my $i += $_ for 1..3; print "localised (1) \$i='$i'\n"; for (1..3) { my $i += $_ } print "localised (2) \$i='$i'\n"; $i += $_ for 1..3; print "not localised (1) \$i='$i'\n"; for (1..3) { $i += $_ } print "not localised (2) \$i='$i'\n"; print "Oops, forgot to zero a widely scoped var!";

      cheers

      tachyon

      s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

Re: 'my' headache...why doesn't this work?
by grinder (Bishop) on Apr 05, 2002 at 15:27 UTC

    It's a bit of a hassle, isn't it. I consider this one of perl's warts. You need a temporary to get the job done, which means you have to do something crufty like:

    #! /usr/bin/perl -wl use strict; my @nums = qw/ 1 2 3 4 5 /; my $sum = do { my $x; $x += $_ for @nums; $x }; print $sum;

    ... possibly localizing $_ in the do block. I sort of understand why things are the way they are, but I'm not good enough at internals to be able to explain in a clear manner :)


    print@_{sort keys %_},$/if%_=split//,'= & *a?b:e\f/h^h!j+n,o@o;r$s-t%t#u'
Re: 'my' headache...why doesn't this work?
by jeffenstein (Hermit) on Apr 05, 2002 at 17:05 UTC

    Deparse is your friend.

    $ perl -MO=Deparse -e 'my $columnsum += $_->[$colnum] for @data;' foreach $_ (@data) { my $columnsum += $$_[$colnum]; }

    The my declaration is within the for loop, so for each @data, a new $columnsum is created.

    Update: As demerphq has pointed out, the deparse output is wrong for this snippet. The todo section at the top of B::Deparse even mentions this specifically. Now I'm off to search for what it is really doing...

    Update2: A google search for "my $x if 0" returns some answers. There is even a node here in the monestary that talks about it.

      Deparse is wrong.

      See my earlier comments to this thread.

      Yves / DeMerphq
      ---
      Writing a good benchmark isnt as easy as it might look.