patgas has asked for the wisdom of the Perl Monks concerning the following question:

I've been struggling to find a non-recursive solution to this, but alas, I have hit the wall. I'm simply trying to loop through all the string combinations from '000' to 'zzzzzzzzzzzzzzzz'. The following code seems to work just fine, but I have a feeling that it'll get ugly once it's dealing with longer and longer strings, hence why I'm looking for the recursion-less fix. If nothing else, I guess I can wrap this up into a package to keep the calling code cleaner. Anyway, here it is:

#!/usr/bin/perl -w use strict; my $number = '000'; do { increment( \$number ); print "$number\n"; } until $number eq 'zzzzzzzzzzzzzzzz'; # or some other long string sub increment { my ( $number_ref ) = @_; my @digits = split //, $$number_ref; unshift @digits, undef while @digits < 16; _increment_digit( \@digits, scalar @digits - 1 ); $$number_ref = join '', grep { defined } @digits; } sub _increment_digit { my ( $array_ref, $index ) = @_; if ( ! defined $array_ref->[$index] ) { $array_ref->[$index] = 0; } elsif ( $array_ref->[$index] ge '0' && $array_ref->[$index] lt '9' + ) { $array_ref->[$index]++; } elsif ( $array_ref->[$index] eq '9' ) { $array_ref->[$index] = 'a'; } elsif ( $array_ref->[$index] ge 'a' && $array_ref->[$index] lt 'z' + ) { $array_ref->[$index] = chr( ord( $array_ref->[$index] ) + 1 ); } elsif ( $array_ref->[$index] eq 'z' ) { $array_ref->[$index] = 0; _increment_digit( $array_ref, ($index - 1) ); } }

"We're experiencing some Godzilla-related turbulence..."

Replies are listed 'Best First'.
Re: Iterating through string combinations
by merlyn (Sage) on Dec 18, 2001 at 03:43 UTC
    my $num = "000"; { $num =~ s{(^|.)(z*)$}{ my ($left, $right) = ($1, $2); $left =~ tr/0-9a-y/1-9a-z/ or $left = "10"; $right =~ tr/z/0/; $left . $right; }e; print "$num\n"; redo while length $num < 20; }
    Be prepared to loop for a very long time.

    -- Randal L. Schwartz, Perl hacker


    update: Yes, delete the ^| up there.
      I think you mean either
      $num =~ s{(.)(z*)$}{
      or
      $left =~ tr/0-9a-y/1-9a-z/ or $left = "1";
        Ahh, no the problem is that I had (.|^) and tested it, but then decided (incorrectly) it should be (^|.). I like your (.) solution the best.

        -- Randal L. Schwartz, Perl hacker

Re: Iterating through string combinations
by mortis (Pilgrim) on Dec 18, 2001 at 03:43 UTC
    If what I think you're doing is correct, your main loop will be attempting to experience 7,958,661,109,946,400,884,391,936 iterations. I say attempting, because I doubt that you (or I, or my children's children) will live to see such a program complete.

    What your incrementation scheme is basicly implementing is a base 36 (26 letters + 10 digits) number. You are incrementing it up to 16 digits, so you'll have 36 ^ 16th iterations (try bc(2) for a quick util that supports arbitrarily large numbers).

    Perhaps there is a better way to attack the problem you're trying to solve? I won't go any further, as this looks not unlike an attempt at cracking passwords.

      Don't worry, it's nothing as malevolant as cracking passwords... While talking to a friend, I had a silly idea to create a Net::AIM bot that worked through all the AOL screen names to figure out the percentages of the 4 user types, and maybe how many screen names are actually still available. I did the math using the same base 36 idea as you, so I realized it was pretty much futile doing it like this. But I still wanted to write the code for it, even if the sun would explode before it finished... *shrug* Maybe in a few years when we have super nano quantum processors, I'll pull it off the shelf and run it then... Anyway, it was an interesting idea, sorta.

      "We're experiencing some Godzilla-related turbulence..."

        If you want to work out percentages, what you *really* want to be doing is to randomly pick valid usernames and go from there.

        The code to randomly generate a username is left as an exercise for the reader. (minus 3 points for saying "Use the loop above to generate an array and then pick random elements from that", although I do grant that is how a mathematician would approach the problem.)

        Q: How does a mathematician boil an empty kettle?
        A: S/He fills the kettle, switches it on and waits for it to boil.

        Q: How does a mathematician boil a kettle full of water?
        A: S/He empties the kettle, reducing it to the previous problem.

        Disclaimer: I may have studied mathematics in the past...

(Ovid) Re: Iterating through string combinations
by Ovid (Cardinal) on Dec 18, 2001 at 03:54 UTC

    If you can't get rid of the recursion, you could always tighten up the program. If you create a mapping of characters to their next character, look how short your final subroutine becomes:

    #!/usr/bin/perl -w use strict; my $number = '000'; my %mapping; @mapping{ 0..9,'a'..'z' } = ( 1..9,'a'..'z','0' ); do { increment( \$number ); print "$number\n"; } until $number eq 'zzzzzzzzzzzzzzzz'; # or some other long string sub increment { my ( $number_ref ) = @_; my @digits = split //, $$number_ref; unshift @digits, undef while @digits < 16; _increment_digit( \@digits, @digits - 1 ); $$number_ref = join '', grep { defined } @digits; } sub _increment_digit { my ( $array_ref, $index ) = @_; $array_ref->[$index] ||= 0; $array_ref->[$index] = $mapping{ $array_ref->[$index] }; _increment_digit( $array_ref, ($index - 1) ) if $array_ref->[$inde +x] eq 'z'; }

    Cheers,
    Ovid

    Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

      Would the following modifications to Ovid's code (above) make it more flexible, or am I deluding myself? I would think then a modification to change sizes would be much easier.

      • Add my $startwith = 3, $endafter = 16; before line 4
      • In line 11, change 'zzzzzzzzzzzzzzzz' to read ('z' x $endafter);
      • In line 18, replace 16 with $endafter

      As to what you wish to do with the code, assuming you are only doing the range (0..9,a..z), you are performing the number of iterations equal to the summation of (36^n) for n from 3 to 16, which is a little less than 7.96e24 (but more than simply (36^16)). To give some perspective to it (if that can be done), if you can perform 1 billion loops per second, you can expect it to take approximately 252 million years to complete them all. Do you really want to do that?

      Comments? Guidance?

Re: Iterating through string combinations
by Zapawork (Scribe) on Dec 18, 2001 at 06:07 UTC
    Well,

    I am working on something similar. For your example you would only need my incrementation loop.. Took me a while to solve the same problem.

    sub forward { while ($cookieforward[0] < 91) { while ($cookieforward[$#cookieforward - $oldforwardcount] > 90 ) { if ($cookieforward[0] > 90) { last } else { $cookieforward[$#cookieforward - $oldforwardcount] = $cookiefo +rward[$#cookieforward - $forwardcount ]; $cookieforward[$#cookieforward - $forwardcount] = &increase($c +ookieforward[$#cookieforward - $forwardcount]); $oldforwardcount = $forwardcount; $forwardcount = $forwardcount + 1; } } $forwardcount = 1; $oldforwardcount = 0; print "Forward Cookie to return:"; foreach $crumb (@cookieforward) { print chr($crumb); } print "\n"; } }
    For me it's a subroutine in a larger forking script. I hope this helps you!

    Dave -- Saving the world one node at a time

Re: Iterating through string combinations
by thor (Priest) on Dec 18, 2001 at 18:39 UTC
    I recently had a similar problem (well, exactly the same problem). The solution I rolled was to come up with an arbitrary base aritmetic solution. That is to say that instead of doing addition base 10, do addition base 128. What I did is to write a function that took an array and made sure that it was in the proper format (all digits were less than 127). If it was not, it would fix it so that it was in the proper format. So, if this function were passed (1,3,129), it would return (1,4,1). The advantage as I saw it was that you could use the normal aritmetic operations without having to worry about anything breaking.

    Once you can do that, you can use map and chr to convert your base-128 vector into a character string. As merlyn alluded to above, no matter how you do it, this problem is exponential. You can be clever and reduce the number of iterations, but it is still exponential.
Re: Iterating through string combinations
by jackdied (Monk) on Dec 19, 2001 at 01:51 UTC
    One, this is a retarted problem to have on the frontpage. Two, whatever action you are going to take with these combinations will consume a ton more time than ANY perl to generate them.

    As someone pointed out, this is very much like cracking passwords, so using the dictionary in Crack and checking those would give you a result. Most dictionary words were taken early on in AOL history, so this would probably give you a skewed result (as would generating a list from any one population).

    -jackdied