spandox has asked for the wisdom of the Perl Monks concerning the following question:

This is a little bit of a question but it may just be a feature request.

Is there a way to, from within a foreach, drop the element you are working with from the array.

Example: @moo = (1 .. 10); foreach (@moo) { if ($_ % 2 > 0) { DROP();} # <----- is what I would LOVE # to do }

Such that the result would be @moo would be (2,4,6,8,10)

Yes I know I can there are all sorts of ways to get the same effect.

Grep: @moo = grep {($_ % 2 > 0)?0:1;} @moo; Or easier to add more complex code if you: @moo = grep {; {if ($_ % 2 > 0){ 0; last; } 1;}} @moo; # PS why do I need that first ';' - I don't know but I do For Loop: for (my $i=0;$i<@moo;$i++) { if ($moo[$i] % 2 > 0) {splice @moo, $i, 1;} } # wow is this slow

But here is what I dislike - the inefficiency of having to use temporary variables or reassaining an entire list.

If a grep could work in place that would be the answer - but it can't. That pretty much doubles the amount of time it takes to work on the array - the assignment back to the original list.

Why is this important to me? I often find myself working with huge arrays. I would love to prune the data in a simple - yet readable away.

Replies are listed 'Best First'.
Re: Foreach for array pruning?
by Zaxo (Archbishop) on Jul 02, 2004 at 18:19 UTC

    I think I prefer your grep solution to the syntax you're asking for. Anyhow, the interior of for (@foo) {} doesn't know what position in the array $_ represents. You can do what you want with the indexes if you take them in reverse order to prevent splice from altering the yet-to-be-seen positions.

    for (reverse 0 .. $#moo) { splice @moo, $_, 1 if $moo[$_] % 2; }
    I imagine that all that splicing is less efficient than grep assignment. Notice that comparison with zero is unnecessary in the conditional.

    This is not too different from your C style loop, but corrects its major flaw.

    After Compline,
    Zaxo

      I imagine that all that splicing is less efficient than grep assignment.

      Definately. using splice like this is horribly inefficient in terms of run time. Every splice requires the entire array to be shifted, and each time with more elements. Which if my hungover brain is correct would run in O(N**2): ie horribly slow.


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi


Re: Foreach for array pruning?
by pbeckingham (Parson) on Jul 02, 2004 at 18:13 UTC

    Recent Perl versions have the ability to delete array indexes as one would delete hash elements.

    my @a = 1..10; for my $i (0 .. 9) { delete $a[$i] if $a[$i] % 2; }
    Please note, however, this operation leaves undef entries unless items are deleted from the end of the array.

Re: Foreach for array pruning?
by dragonchild (Archbishop) on Jul 02, 2004 at 18:34 UTC
    If a grep could work in place that would be the answer - but it can't. That pretty much doubles the amount of time it takes to work on the array - the assignment back to the original list.

    Are you sure that the 5.8.x series does this? I was under the impression that the same map optimizations were applied to grep when assigning to the same list.

    Not to mention, why on earth are you trying to optimize grep? This smells like a case of premature optimization ...

    ------
    We are the carpenters and bricklayers of the Information Age.

    Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose

    I shouldn't have to say this, but any code, unless otherwise stated, is untested

Re: Foreach for array pruning?
by diotalevi (Canon) on Jul 02, 2004 at 18:00 UTC
    splice @ary, $_, 1 for reverse grep ! $ary[$_] % 2, 0 .. $#ary;
    splice @ary, $_, 1 for grep ! $ary[$_] % 2, 0 .. $#ary;

    Drat. That'll screw up because all the indexes would be wrong.

Re: Foreach for array pruning? (Updated)
by BrowserUk (Patriarch) on Jul 02, 2004 at 19:55 UTC

    Updated conclusion:

    You can save some ram (assuming you haven't got any to spare) at a penalty of 10% runtime.

    my @array = 1 .. 1_000_000; ## Note the above doubles mem. consumption. ## Half of which is immediately availble to the rest of the program. # Ripple the good values through the array my( $s, $d ); for( $s = $d = 0; $s < @array; $d++, $s++ ) { $array[ $s ] %2 and $s++;; $array[ $d ] = $array[ $s ]; }; # and truncate it. $#array = --$d;

    Memory test code.

    Benchmark

      I think that misses the point. It was more a general question with a "trivial" example

      BTW I was on perl 5.6.1
      I installed 5.8.4 (and a boatload of modules - thanks CPAN) and did a time comparison

      Code used: #!/usr/bin/perl use Time::HiRes qw(gettimeofday tv_interval ); use strict; # good habits start at birth my ($start,$stop); my @moo; @moo = (1 .. 1000000); $start = [gettimeofday()]; @moo = grep {; {if ($_ % 2 > 0){ 0;last; } 1;}} @moo; $stop = [gettimeofday()]; print tv_interval($start, $stop), "\n\n";

      results:
      5.6.1 did pretty much the same as 5.8.4

      PS: anyone figure out why I needed the ';' in between the squigle brackets for the code blocks?

        I'm not quite sure what you mean by "misses the point"?

        I thought you were looking for a way to avoid your percieved inefficiencies of using grep to filter large arrys. The only 'inefficiency' that really exists is that (notionally) a list is created on one side of the grep which is then filtered and assigned back to the array on the other. Ie. Perl trades space for time.

        Given the way Perl implements it's arrays, there is no trivial way of removing an element from the middle.

        [@myarray]-->[ 0 ]->[ value 1 ] [ 1 ]->[ value 2 ] [ 2 ]->[ value 3 ] [ 3 ]->[ value 4 ] [ 4 ]->[ value 5 ]

        To remove a value from the middle (say element 2) in-place, you have to ripple the pointers.

        [@myarray]-->[ 0 ]---->[ value 1 ] [ 1 ]---->[ value 2 ] [ 2 ]-\ [ 3 ]-\\->[ value 4 ] \->[ value 5 ]

        In effect (if not in exact implementation), is to copy pointers from the old list into a new one omiting those that fail the test. Then discard the old array if the destination is the same as the source. Very efficient cpu-wise but does require a extra memory.

        Doing it in-place avoids the extra memory, but there is no way to void the ripple. That's basically what my snippet did. It's obviously not going to be as fast, though I was surprised how little slower it was. Whether it would be possible to add an optimisation to grep to do this in line if the dest == src I'm not sure. I know that sort has a similar 'in-place' optimisation for dest == src invocations.

        With respect to the need for the ';'... It's required to allow the parser to determine that your inner code block

        @moo = grep {; {if ($_ % 2 > 0){ 0;last; } 1;}} @moo;

        Is a code block and not an anonymous hash. More typically this is done with a unary '+'

        @moo = grep { +{if ($_ % 2 > 0){ 0;last; } 1;} } @moo;

        Alternatively, you could avoid the need for the inner block completely and do

        @moo = grep { not ( $_ % 2 ) } @moo;

        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "Think for yourself!" - Abigail
        "Memory, processor, disk in that order on the hardware side. Algorithm, algoritm, algorithm on the code side." - tachyon
        PS: anyone figure out why I needed the ';' in between the squigle brackets for the code blocks?

        At a guess I'd say it's so perl can disambiguate between a block of code and the start of a hash ref construction.

Re: Foreach for array pruning?
by CombatSquirrel (Hermit) on Jul 02, 2004 at 19:54 UTC
    I encountered the same problem on a project of mine. The solution I came up with to avoid manipulating the loop variable in a for-loop was processing the whole array backwards. This might or might not help you:
    #!perl use strict; use warnings; my @foo = (0..9); for (reverse(0..$#foo)) { splice @foo, $_, 1 if $_ % 2; } print join ' ', @foo;
    Hope this helped.
    CombatSquirrel.
    Update: Argghh, Zaxo was faster than I...

    Entropy is the tendency of everything going to hell.