in reply to Any thoughs on Iterator prefetching?

Looking through the code, the author states:

From _initialize (called from new):
# Caches the first value of the iterator in %next_value +_for.
and value:
# Notes: Keeps one forward-looking value for the iterator +in # %next_value_for. This is so we have something + to # return when user's code throws Am_Now_Exhauste +d.
Assuming this is to know when the iterator will be exhausted, as pKai said.

For your second example, this is explicitly stated in the docs:
When you use an iterator in separate parts of your program, or as an a +rgument to the various iterator functions, you do not get a copy of t +he iterator's stream of values. In other words, if you grab a value from an iterator, then some other +part of the program grabs a value from the same iterator, you will be + getting different values. This can be confusing if you're not expecting it. For example: my $it_one = Iterator->new ({something}); my $it_two = some_iterator_transformation $it_one; my $value = $it_two->value(); my $whoops = $it_one->value; Here, some_iterator_transformation takes an iterator as an argument, a +nd returns an iterator as a result. When a value is fetched from $it_ +two, it internally grabs a value from $it_one (and presumably transfo +rms it somehow). If you then grab a value from $it_one, you'll get it +s second value (or third, or whatever, depending on how many values $ +it_two grabbed), not the first.
IMO, if you needed to use the value returned from the iterator, save it somewhere instead of using a variable in the same scope - although I realize that your code is more than likely a simplification of what you're really trying to do

Replies are listed 'Best First'.
Re^2: Any thoughs on Iterator prefetching?
by mzedeler (Pilgrim) on Jun 25, 2009 at 20:52 UTC

    Thanks for pointing out that the design is intentional. The example from the documentation doesn't address the buffering issue, I am concerned with. A bufferless implementation would behave the same way. Try running this minimal test:

    use Test::More tests => 1; use Iterator; my $value = 0; my $iterator = new Iterator(sub { return ++$value }); is($value, 0, "Haven't called the iterator yet.");

    The reason that I have raised this question is because I'd like to discuss the pros and cons of this design. What exactly is it that you get out of having a buffering iterator?

    (I know that I have submitted a bug on this behaviour which was before I realized that the behaviour was intentional.)

    If the price of being able to tell whether the iterator has been exhausted or not is buffering, then the price in terms of problems dealing with this buffering is way too high.

    Another consideration is whether there is enough reason to implement iterators in a separate class, since perl offers closures that can be used to do almost everything Iterator has to offer.

    All in all I want to discuss pros and cons of various iterator designs.