Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

How to flatten a list of lists?
This is what I have been playing with.
It seems to work, but I wonder: Is there an easier/better way?

sub flatten { my ($LoL) = @_; my @flat; for my $arr (@$LoL) { if ( ref $arr ) { push(@flat,flatten($arr)); } else { push(@flat,$arr); } } return @flat; } my @arr = ([1,2],[3,4,[5,6]],7); my @list = flatten(\@arr); print "@list\n";

L

Replies are listed 'Best First'.
Re: flattening a list-of-lists
by broquaint (Abbot) on Nov 19, 2003 at 12:24 UTC
    Depends on your definition of 'easier' and 'better' really since that's a nice solution (although it should probably check the return of ref). Perhaps ...
    sub flatten { map { ref $_ eq 'ARRAY' ? flatten(@$_) : $_ } @_ } print join ', ' => flatten 1, 2, [3,4], [5, [6,7, [8,9] ],10]; __output__ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
    But we're just golfing now.
    HTH

    _________
    broquaint

Re: flattening a list-of-lists
by Abigail-II (Bishop) on Nov 19, 2003 at 12:27 UTC
    "Easier" and "better" are subjective. But it can certainly be shorter:
    sub flatten; sub flatten {map {ref $_ ? flatten @$_ : $_} @_}

    Abigail

Re: flattening a list-of-lists
by Limbic~Region (Chancellor) on Nov 19, 2003 at 18:24 UTC
    Anonymous Monk,
    Here is a non-recursive approach that may be more flexible. It will not follow the same array ref twice, which will prevent circular references causing infinite loops.
    #!/usr/bin/perl -w use strict; my @array1 = ( 1 .. 10 ); my @array2 = ( qw(a b c) , [ 28, 42, 84 ] , 'foo bar' ); push @array1 , \@array2; push @array2 , \@array1; print join ' , ' , flatten(\@array1 , \@array2); sub flatten { my @stack = @_; my @flat; my %seen; $seen{$_} = 1 for @stack; while ( @stack ) { my $array = shift @stack; for my $element ( @$array ) { if ( ref $element eq 'ARRAY') { if ( ! $seen{$element} ) { $seen{$element} = 1; unshift @stack , $element; } } else { push @flat , $element; } } } return wantarray ? @flat : \@flat; }
    Cheers - L~R

    Update: I asked for comments in the CB since this is the first time I have tried doing this:

    • demerphq - ref can be fooled by blessed arrays, better to use Scalar::Util::reftype()
    • demerphq - Even that is prone to problems when overloaded stringification is present
    • tye - It is assumed that @stack will only contain array refs. Code to handle for mistakes should be added

      Hi Limbic~Region, in response to your question in the CB I wrote the following code. It handles the issues you mention in your update as well as the fact that you are doing the wrong kind of traversal.

      The output I get for the two variants is below.

      flatten:1, 2, 3, 4, 5, 6, 7, 8, 9, 10, a, b, c, 28, 42, 84, foo bar lr_flatten:1, 2, 3, 4, 5, 6, 7, 8, 9, 10, a, b, c, foo bar, 28, 42, 84 * flatten:bar, baz, 2, foo, 1, baz, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, a, + b, c, 28, 42, 84, foo bar

      The star result is from dumping a hash, something I added in just for the hell of it. :-)

      Cheers,


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi


        Updated: It now includes checking for cycles.

        That seems way complicated. I went with:

        *isa = \&UNIVERSAL::isa; sub flatten { my @stack= reverse @_; my %seen; my @out; while( @stack ) { my $value= pop(@stack); if( !ref($value) || !isa($value,"ARRAY") ) { push @out, $value; } elsif( ! $seen{0+$value}++ ) { push @stack, reverse @$value; } } return @out; } sub flatten2 { my @out; my %seen; while( @_ ) { my $value= shift(@_); if( !ref($value) || !isa($value,"ARRAY") ) { push @out, $value; } elsif( ! $seen{0+$value}++ ) { unshift @_, @$value; } } return @out; } my @list = ( 1, [ 2, 3 ], 4, [ [ 5, 6 ], 7, [ 8 ] ] ); print join " ", flatten( @list, @list ); print $/; print join " ", flatten2( @list, @list ); print $/;
        which outputs:
        1 2 3 4 5 6 7 8 1 4 1 2 3 4 5 6 7 8 1 4

                        - tye
Re: flattening a list-of-lists
by zby (Vicar) on Nov 19, 2003 at 12:32 UTC
    What you are flattening here is a tree, not a list of list - you do a depth-first walk on it. It's a standard algorithm - not much to improve there (beside, perhaps, some tail-recursion tweaking - I've never cared to learn about that technique).
      as zby pointed out...you're doing a tree search/flatten. a standard flattening of arrays in perl is simply assignment  @flat = (@this, @that, @the_other);...where the flat array now contains all elements of the assigned arrays, not pointers etc, and there is no way to re-create the original three arrays (without cheating) from the final flat array.
Re: flattening, No recursion
by shotgunefx (Parson) on Nov 19, 2003 at 20:10 UTC
    sub flatten { my %seen; my @temp = @_; for (my $i=0; $i < @temp; $i++){ if ( UNIVERSAL::isa($temp[$i], 'ARRAY') ){ die "Circular reference! Bailing..." if $seen{$temp[$i]}++; splice (@temp, $i, 1, @{$temp[$i]} ) } } return wantarray ? @temp : \@temp; }


    -Lee

    "To be civilized is to deny one's nature."
Re: flattening a list-of-lists
by calin (Deacon) on Nov 19, 2003 at 14:36 UTC
    1-level deep flattener which uses string eval.

    sub flatten_stringeval { eval join ',', map {ref $_[$_] eq 'ARRAY' ? "\@\{\$_[$_]\}" : "\$_[$ +_]"} (0..$#_); }

    This exemplifies the automatic flattening of arrays in perl list literals.

      This is an extremely scary solution. So scary that I can only assume you are joking. Please dont suggest things like this without a security disclaimer. A fresh newbie might think its a good plan then end up having their hard drive deleted or other nasty tings. And security issues aside its going to be many times slower than a subroutine based solution.

      Gee.
      Do I feel like merlyn is standing behind me or what? :-)


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi


        Well, to be truly like merlyn, you would now need to post the following test case to catch people on defensive programming style too.

        my @a = \@a; flatten(\@a);

        Yes, merlyn got me on this one once in a similar thread :-)

         

        perl -le "print+unpack'N',pack'B32','00000000000000000000001010010000'"

        I wanted to make a point. Show what Perl does when you feed it code, by writing code that writes code and feeds it back to Perl, as if it were written by you :) . Of course, the obvious solution is to just take away recursion from broquaint's version:

        sub flatten { map { ref $_ eq 'ARRAY' ? @$_ : $_ } @_ }

        This is an extremely scary solution. So scary that I can only assume you are joking. Please dont suggest things like this without a security disclaimer. A fresh newbie might think its a good plan then end up having their hard drive deleted or other nasty tings.

        This sounds freaky. I'm curious, can you post an exploit against flatten_stringeval?. The only thing interpolated is $_, which iterates through (0..$#_). Everything else is quoted. Also, it checks for hard array ref before putting @{...}, so one can't play games with symbolic references (one doesn't need string eval for that, just no strict 'refs'). Maybe I'm missing something...

Re: flattening a list-of-lists
by Roger (Parson) on Nov 20, 2003 at 00:45 UTC
    I have thought about a bruteforce method to flatten a deeply nested list with Data::Dumper, by massaging the output of Data::Dumper. It doesn't have any recursion (at least not in my perl script), and all that's needed is a simple regular expression to strip out unwanted stuff. I have quickly wrote the follow code to prove that it can be done. Forget about the performance of this method, it's just a proof of concept. :-)

    #!/usr/local/bin/perl -w use strict; use Data::Dumper; my @arr = ([1,2],[3,4,[5,6]],7); my $list = flatten(\@arr); print Dumper($list); sub flatten { my $data = Dumper(shift); $data =~ s/(\$\w+\s+=\s+)|[\n\[\];]//gm; # massage the output return [eval "$data"]; }
    And the output is as expected -

    $VAR1 = [ 1, 2, 3, 4, 5, 6, 7 ];