in reply to Re: The Anomalous each()
in thread The Anomalous each()

Just thinking out loud here with some untested code...

sub each_iter(%) { my $hash = shift; my @keys = keys %$hash; sub { my $nkey = shift @keys; $nkey => $hash->{$nkey}; } } my $iter = each_iter(%some_hash) while (my ($k, $v) = $iter->()) { # do stuff }
This kind of reminds me of Writing Solid Code's rewrite of realloc. Any use of realloc would be "code smell" as well, so Steve Maguire diligently set out to prove it, then rewrite it safely. If it stinks so bad, what can we do to provide a safer version? The above is probably as good as it gets without XS code. Perhaps if someone has a Hash::Util module to stick this into... Anyway, just because a language comes with a function doesn't mean it's always the best. At the time, each was probably quite nifty. So was realloc. Hindsight is 20/20, and now we probably know better. Which is why, as I recall, perl 6 will do this just a wee bit differently.

Update: I agree with Aristotle that it's not even close to ideal. However, I'm not sure it's "not very helpful". For small hashes (for some definition of small which may depend on free memory), it can indeed be incredibly helpful: you can run the each iterator inside itself - i.e., two loops that absolutely must run inside each other. You can't do this without taking a copy of the keys anyway, so this provides the syntactical sugar to do it the each way that you want. It also provide a small-change to existing code (create the iterator on the previous line, change the each call to $iter->()) which you want to get working quickly. It means you can get going again quickly. Personally, I almost never use each to begin with - probably from this exact same problem. So it's not as helpful to me, but I could see reasons to use this function.

Besides - I put it out there more to try to prompt someone with the XS skills to try it than to propose it as a "good" solution".

Replies are listed 'Best First'.
Re^3: The Anomalous each()
by Aristotle (Chancellor) on Nov 19, 2005 at 06:28 UTC

    There is a Hash::Util, which is core in 5.8. There also is Hash::MoreUtils (although it doesn’t look like nearly as useful an addition as its cousin List::MoreUtils is).

    I agree that having an iterator maker function handle this is the better idea, but doing it in pure Perl is not very helpful. The point of each is to avoid building a huge list – f.ex., when you’re dealing with very large DBM files, you really don’t want to use keys on them.

    The code you show does not achieve this. I can’t think of any case in which I’d prefer it over a straight foreach( keys %hash ) (not even to pass the iterator around – I can just as well pass an array of keys around).

    If anyone has the XS chops to take this on, it would be good idea.

    Makeshifts last the longest.

Re^3: The Anomalous each()
by Aristotle (Chancellor) on Nov 19, 2005 at 11:10 UTC

    For small hashes (for some definition of small which may depend on free memory), it can indeed be incredibly helpful: you can run the each iterator inside itself – i.e., two loops that absolutely must run inside each other.

    But you can do that just as well with two nested foreach( keys %hash ) loops. There is nothing you can do with your function that cannot be done more straightforwardly with direct use of keys. each is necessary only when memory is a concern; your solution does not provide for that.

    Makeshifts last the longest.

      I'm sorry that you seem to miss the point of the update. There is a lot of stuff in perl that isn't necessary. We don't need for, foreach, and while. We can get away with just while. The others provide some really nice syntactical sugar, though, for common cases. And they provide a way for someone else (the folks who write the code for perl itself) to optimise these common cases. Once people start using an iterator version of each, optimising it becomes much more justifiable.

      I'm also sorry that you seem to not be able to think of places where this idea is useful as a quick-fix. For example, in the C++ code that some of my teammates write and maintain, there are a number of containers. The guy who wrote them created a "getNextObject()" method for many of them so that you could call "for (Object* po = container.getNextObject(); po != NULL; po = container.getNextObject() { ... }" to iterate through the container. (He was the design lead for the code and declared STL iterators verbotten - I ignored this stupid decree when I wrote my containers.) Six months later (doesn't everything bad happen to your code six months later?), they tracked down a bug to where two functions were doing this to the same container, and one function was now calling the other. We switched it out for an interator that had nearly the same syntax, made minor changes to the calling code, and fixed the problem. Well, it was more of a workaround because they're kind of bastardised versions of iterators.

      In perl, the best we can do without XS and messing with perl's guts, is the above perl code if we need to suddenly change some functions to stop using each because of nesting that may not have been occurring when we originally wrote the code. For some people, rewriting dozens of uses of each to use keys may seem like too large of a change to the code, whereas using an iterator that otherwise feels like each may seem like the code is changing as little as possible.

      It's syntactical sugar. That's all. I'm not sure why you feel the need to so vehemently attack syntactical sugar. If you examine the ingredient list on the box that perl comes in, I'm pretty sure syntactical sugar will come in somewhere in the top 5 ingredients. Once implemented in pure perl, it becomes much more straightforward for an XS wizard to come along and provide a faster, more memory-efficient version. Straightforward in the way that anyone using the pure perl version should be able to just swap out one for the other. If this function were added to Hash::MoreUtils today, then someone could put the XS version in tomorrow, and the code written today would not need any changes tomorrow to reap those benefits. This function merely is providing a stepping stone to getting there.

      Update: Added italicised part to emphasise multiple uses of each as Aristotle seems to have been interpreting too literally. I'm also a bit unclear on why he's comparing while ( my $k = each %hash ) { ... } when as far as I can tell, using each for just the keys is a rare occurance. In fact, BrowserUk's follow-on thread was the first time I had ever seen that. Not a very realistic comparison.

        There is a lot of stuff in perl that isn’t necessary. We don’t need for, foreach, and while. We can get away with just while.

        The examples you bring up offer mental shortcuts for specialised scenarios.

        Now observe:

        while( my $k = each %hash ) { ... } foreach my $k ( keys %hash ) { ... }

        Except for the added pitfall of each, these are completely identical. each is no mental shortcut for keys, or vice versa. It merely exposes an implementation detail for when you need to know about it. In Perl6, there won’t even be any sort of difference, and all these constructs will use the same syntactic constructs. Without the difference in implementation detail, any distinction is pointless.

        For some people, rewriting to use keys may seem like too large of a change to the code, whereas using an iterator that otherwise feels like each may seem like the code is changing as little as possible.

        You can rewrite

        while( my ( $k, $v ) = each %hash ) {

        as either

        foreach my $k ( keys %hash ) { my $v = $hash{ $k };

        or

        sub each_iter(%) { my $hash = shift; my @keys = keys %$hash; sub { my $nkey = shift @keys; $nkey => $hash->{$nkey}; } } my $iter = each_iter %some_hash; while( my ( $k, $v ) = $iter->() ) {

        Which one is least invasive is your call.

        Makeshifts last the longest.