I don't care about "cool". I care about "useful". Callbacks are fundamentally inflexible (Re: Are you looking at XML processing the right way? (merge)).

Note that turning my iterator into your callback interface is trivial:

sub forDerange(&@)
{
    my $cv= shift @_;
    my $iter= genDerange( @_ );
    my @list;
    while(  @list= $iter->()  ) {
        $cv->( @list );
    }
}
[download]

Try to go the other way. (:

It's not too difficult to go the other way using Recursively-generated Iterators.

---

Caution: Contents may have been coded under pressure.

I didn't have to rewrite my iterator, just use it. I believe you are pointing out ways to make it easier to rewrite a naive recursive solution so that it can be re-implemented as an iterator instead.

By "Try to go the other way" I meant (at least in part), try to use jdporter's callback interface (as-is) in a situation where an iterator is needed.

Though, I don't even understand what you linked to yet so I still consider what I did to go from iterator to callback to be "trivial" compared to what you are proposing as "not too difficult". (:

- tye        

Towards the best collection traversal interface

Most programming languages support collections, represented by an in-memory data structure, a file, a database, or a generating function. A programming language system gives us typically one of the two interfaces to systematically access elements of a collection. One traversal API is based on enumerators -- e.g., for-each, map, filter higher-order procedures -- of which the most general is fold. The second approach relies on streams, a.k.a. cursors, lazy lists. Generators such as the ones in Icon, Ruby and Python are a hybrid approach.

It is well-known that given a cursor interface to a collection, we can implement an enumerator. It is less appreciated that given an enumerator interface, we can always derive a cursor -- in an automatic way. We demonstrate that generic procedure for languages with and without first-class continuations.

Now that cursors and enumerators are inter-convertible, an implementor of a collection has a choice: which of the two interfaces to implement natively? We argue that he should offer the enumerator interface as the native one. The paper elaborates that enumerators are superior: in efficiency; in ease of programming; in more predictable resource utilization and avoidance of resource leaks. We present a design of the overall optimal collection traversal interface, which is based on a left-fold-like combinator with premature termination. The design has been implemented and tested in practice.

Only having read the two abstracts, it seems to me that the authors rely on a Scheme-like language that has continuations or at least coroutines. With coroutines, it's trivial to convert between an "enumerator" (callback) and a "stream" (list/iterator). Without a coroutine mechanism, it's not as easy. As you seem to have read the paper, can you maybe post a link to the actual paper or, even better, give an application of the automatic way discussed there in Perl?

Using the Coro.pm module, it's quite easy to have generators and to convert between enumerator and iterator, but Coro has the disadvantage of relying on a GPLed library and it doesn't (immediately) work on Win32. Without Coro, one has to manage the stack oneself and/or to create a large buffer for all values passed by the enumerator from what I know. But maybe the paper shows a technique I don't know (yet).

Update: I had another look at Coro, and it isn't under the GPL. I also found that half-support for Win32/MSVC is there, now. Yay! ;)

Update If one needed to allow for deranging a list which contains duplicates, one could simply derange the list of its indices.

Um, not really. A derangement algorithm that "doesn't handle duplicates" is one that deranges "1 4 4" into "4 1 4" because it doesn't notice that the duplicates are the same and so doesn't realize that replacing the last 4 with the second 4 didn't actually cause the last item to be different. It also, when deranging "1 1 2 2" returns "2 2 1 1" four times instead of just once because it doesn't realize that reversing "1 1" (nor "2 2") gives the same thing.

So your first algorithm simply breaks when presented with duplicates (it always finds zero derangements, for those who didn't notice) but your work-around just prevents this breakage while not actually correctly handling the duplicates. (So I'd suggest you document how it doesn't handle duplicates rather than suggest that work-around.)

Just for fun, here is your algorithm modified to correctly handle duplicates:

sub _derange
{
    my( $cb, $av, $todo, @i ) = @_;
    return $cb->( @$av[@i] )
        if  ! @$todo;
    my( %iseen, %vseen );
    @iseen{@i}= ();
    @vseen{@$av[@i]}= @i;
    my( $range, @todo )= @$todo;
    for(  @$range  ) {
        _derange( $cb, $av, \@todo, @i, $_ )
            if  ! exists $seen{$_}
            and     ! exists $vseen{$av->[$_]}
                ||  $vseen{$av->[$_]} < $_;
    }
}

sub derange(&@)
{
    my $cb= shift @_;
    _derange(
        $cb,
        \@_,
        [ map {
            my $x = $_[$_];
            [ grep { $_[$_] ne $x } 0..$#_ ]
        } 0..$#_ ],
    );
}

derange( sub { print "@_\n" }, 1,1,2,2,3 );
[download]

Of course, if you try to look up a definition for "derangement", you won't find anything that makes much sense when considering duplicates because mathematicians define derangements in term of permutations which they define without considering duplicates either (though they usually don't use language that actually makes that clear, either).

But the extension of these two concepts to cover lists with duplicate elements is natural, even obvious, as well as useful.