And now I took longer to get back to you. :-(

First of all I hope that I again stimulate similar theoretical discussions to what I did before. I have a somewhat unusual approach to theory, namely I don't feel that I understand a theory unless I can integrate it into my own decision making process. And I have found that actually attempting to explain what I believe, and why I believe it, then comparing that to other people's points of view strongly helps me to achieve that integration (or else to decide that the theory doesn't work for me).

In other words I engage in these discussions because I learn, and the fact that some others like them is just icing on the cake. :-)

Furthermore I agree with you on the value of not just accepting orthodoxies, but understanding their limitations. For instance see Re (tilly) 1: The qq{worse is better} approach (discussion) for me arguing the value of developing your own understanding, and see Re (tilly) 2: Paradigm Shift - Don't use strict to see a thread where I discuss (among other things) when goto is justified, and why it is less often justifiable in Perl than C.

About the use of the word "trick" versus "technique" - you are right, had I used "technique" we might not have had this discussion. And there is no real difference - one person's trick is another's technique. However I used the word "trick" for a reason. That reason is that I wanted to underscore the difference between techniques which only buy you a bit in a specific context and ones which you tend to win from over and over again. Now what is a specific context? Well that depends what you do - one programmer's common situation is another's rare case that they never encounter.

Thus I (like any reasonable human) tend to focus on the situations that I have encountered often. I have not often encountered cases where (at least with a little thought) one has to be overly concerned with running out of memory. Where one does, it has generally been fairly easy to address. However I have dealt with many where it is very convenient to grab all data into RAM, and operate in passes - one builds a data structure, another dumps that data structure, etc. In that case there is no real disadvantage to foreach, while if I wanted to hammer home the benefit of while I would also need to make people aware of when they have to use explicit iterative approaches, and can't just pass the entire dataset to a map or function call (because it forces you to suck it all into RAM).

If the problems I was dealing with were ones where you did constantly worry about running out of room (for instance validation routines on large databases), then I would be more sympathetic to the resulting issues. I undoubtably would decide that a different circle of programming techniques is optimal, and would focus differently.

The moral being that programs exist in an environment, and the effectiveness of programs and programming techniques is the result of an interaction between problems, machines, users, and other programmers. Given that trade-offs are inevitable, the right trade-offs for one situation frequently are counterproductive for another. Which is fundamentally why it is important to learn to question orthodoxies - because you need to come up with effective trade-offs for your situation. And you can't do that if you look at answers as something descended from Heaven rather than as something which you can (through analysis) produce appropriately for your needs. Of course willingness to question doesn't guarantee the skill - it is merely a necessary precondition to developing it. (It is also a guarantee of a few burned fingers as you learn firsthand why, for instance, you don't touch hot stoves... :-)