QM has asked for the wisdom of the Perl Monks concerning the following question:

I've been tinkering with combinations of strings from different arrays, and discovered that the ubiquitous glob solution breaks for long input strings.

Update: The below example is for Strawberry Perl 5.16.1 on Windows. On Ubuntu 10.4, Perl 5.10.1 (i486-linux-gnu-thread-multi), it first breaks at 4027.

Here's a trivial example to show the effect: 

#!/usr/env/perl

use strict;
use warnings;

my $q = (shift or 129); 
my @x = ( 'A' x $q, 'B' x $q );

my $glob = '{' . join(',', @x) . '}';

while (my $g = glob($glob)) {
    print "$g\n"; # no output if $q > 129
}

##### returns nothing with 129 ####

[download]

What I really want to do is more like this, but glob also fails when longer strings are used: 

my @x = (1..10);
my @y = qw(one two three four five six seven eight nine ten);
my @z = qw(Amy Bob Cal Deb Eck Fay Gin Hob Ian Jan);

my @xyz = (\@x, \@y, \@z);
my $glob = join( ',', map { '{' . join( ',', @$_ ) . '}' } @xyz );

[download]

Q1: Why does glob quietly fail?

Q2: Is there a built-in, core, or module that is not so limited?

Instead of that, I suspect I'll have to do something like the following. This is very simplistic, but does the job. (I'm sure this has been done before, but I haven't spent enough time searching.)

Q3: Any comments here? 

sub make_combo_iter
{
    my $array_ref = shift;
    
    # last index of top level
    my $size = scalar( @{$array_ref} );
    
    # create array of last indices
    my @array_limits;
    for my $i (0..$size-1)
    {
        $array_limits[$i] = scalar( @{$array_ref->[$i]} ) - 1;
    }
    
    # one counter for each sub-array
    my @array_counter = (0) x $size;
    my $done = 0;
    my $iter = sub { 
        if ($done)
        {
            $done = 0;
            return undef;
        }
         
        my @next_combo;
        # compute @next_combo
        for my $i (0..$size-1)
        {
            push @next_combo, $array_ref->[$i][$array_counter[$i]];
        }
         
        # increment counters, set $done if the last one rolls over
        my $rollover = 0;
        for my $c (0..$size-1)
        {
            if (++$array_counter[$c] > $array_limits[$c])
            {
                $array_counter[$c] = 0; # reset
                $rollover = 1;          # carry
            }
            else
            {
                $rollover = 0;  # no carry
                last;           # no more increments
            }
        }
        
        # if the last array rolled over, set the done flag
        if ( $rollover )
        {
            $done = 1;
        }
         
        return \@next_combo;
    }; # anonymous iterator sub
    return $iter;
} # make_combo_iter

[download]

-QM
--
Quantum Mechanics: The dreams stuff is made of

Replies are listed 'Best First'.
Re: glob quietly fails
by daxim (Curate) on Oct 15, 2012 at 13:21 UTC
[reply]
      see Re: Hexamer combination from 3mers. about Algorithm::Combinatorics.

      Understood. But that doesn't solve the "array of arrays, 1 from each" problem. A shorter version of the OP glob spec would be 

      @x = (1..2);
@y = qw(one two);
@z = qw(Amy Bob);

@xyz = (\@x, \@y, \@z);
$glob = join( ',', map { '{' . join( ',', @$_ ) . '}' } @xyz );
print "$glob\n";

# prints '{1,2},{one,two},{Amy,Bob}'

      [download]

      I don't know of a module that can emulate glob on that.

      -QM
      --
      Quantum Mechanics: The dreams stuff is made of

[reply]
[d/l]
[select]

        Actually I think your iterator solution is not half bad. It could work very well where nested foreach loops are not an option.

        I'd drop the redundant array_limits array, however.

[reply]
Re: glob quietly fails
by Anonymous Monk on Oct 15, 2012 at 15:06 UTC
[reply]
[d/l]
Re: glob quietly fails
by RichardK (Parson) on Oct 15, 2012 at 18:08 UTC

    What are you expecting the first code example to do?

    As it is, it creates an array of 2 strings each 129 characters long and then glob returns them. Which is working just fine for me ;)

    UPDATE : Ah! your calling glob in scalar context! try a list context instead.

    for my $g (glob(...) ) {
}

    [download]
[reply]
[d/l]

      Hello RichardK.

      glob fails silently as QM saids with my perl. How does this code say with your platform? 

      for (my $q =1; $q < 10000; $q++){
    my @x = ( 'A' x $q, 'B' x $q );
    my $glob = '{' . join(',', @x) . '}';

    my $g = join(',', glob($glob));
    if (length $g == 0){
        print "fail silently, q=$q\n";
        last;
    }
}

      [download]
      Mine stops saying 
      fail silently, q=511
      perl -v 
      This is perl 5, version 12, subversion 2 (v5.12.2) built for i386-freebsd-thread-multi-64int
      regards.

[reply]
[d/l]

      UPDATE : Ah! your calling glob in scalar context! try a list context instead.

      Both are supposed to work

[reply]
      As remiah and Anonymous Monk have stated, I expected while and for to behave the same way, and they do for me, on Strawberry Perl v5.16.1: 
      #!/usr/env/perl

use strict;
use warnings;

my $q = (shift or 129); # 128 is the largest working value
my @x = ( 'A' x $q, 'B' x $q );

my $glob = '{' . join(',', @x) . '}';

while (my $g = glob($glob)) {
    print "while: $g\n"; 
}

for my $g ( glob( $glob ) ) {
    print "for: $g\n"; 
}

__EOF__
#################
# result:
#
while:
for:

      [download]

      Running this with a command line argument of 128 gives the expected output of 2 very long strings for while and 2 for for.

      -QM
      --
      Quantum Mechanics: The dreams stuff is made of

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom