I believe that the goal of programming is to solve problems, not to produce code. If you can solve problems without writing code, so much the better.

The single biggest contribution to reliability, security and maintainability, is using less code to solve the problem.

Note: using, not writing. You may succeed in writing less code by using a one or more modules, but if the modules are over-engineered, too generic, grossly hierarchical, or otherwise verbose, then you can still end up with more code than you need. And that's bad all round.

You're right, though I do believe that the far more frequent sin is writing too much code, not reusing too much code.

Here is where I make a clear distinction and was happy that you wrote this node. I very seldom have the hubris to believe I have created something superior. I go back and examine very closely existing wheels. I run my code through their test suites. I discover what bad assumptions I made and how others addressed them. I spend way more time on a problem than is required to solve it so I might squeeze the very essence out of it and learn as much as it has to offer.

This isn't the only way that I "waste" my time learning. I spend a lot of time answering questions here and on #perl IRC. I see the common elements of problems over and over again. I see how others responding solve the problem. I learn from what they have to offer and often will summarize the differences in the solutions and why one may be more advantageous over the others. It has been my experience that well worn wheels are able to accomplish the job for 98% of the general problems - there is still 2% that require going beyond existing wheels.

I spend this 75% of the time doing "pure" research so that I can do my best the 25% of writing "applied" solutions. I do make money from programming (hobby vocation) and I do solve real world problems (for myself, for my employer and for complete strangers). This includes very large complex applications and not just 50 line scripts.

After nearly 7 years of programming in perl, I think I would still be a novice if I only wrote code when I needed to.

This isn't the only way that I "waste" my time learning.

Learning isn't a waste of time, not in and of itself. Undisciplined learning is.

I'll try a different metaphor. Playing scales and arpeggios on the piano (or Chopin's Études) is usually a good exercise, but if you slouch or rest your wrists on your knees and have poor fingering technique, they're lousy, wasted exercises. If you let yourself slip up and hit the wrong keys, you're only hurting yourself.

In a similar way, well-focused research is often valuable. I know you make a sharp distinction (as do I) between experimental, research code and code you intend to maintain. That's a very important point. If discipline in practice and research is important (and it is), it's because it helps to establish and maintain good habits for when you use your skills for real.

chromatic,
Learning isn't a waste of time, not in and of itself.

I only meant waste in the sense of throw away code with no immediate tangible benefit and not in the sense that there was no value. Putting quotes around it didn't do an adequate job so I will draw on another analogy.

A friend of mine who is an artist in the traditional sense (painter) told me that his really good stuff only comes when he is inspired by the muse. This was after telling him I was spending less and less time playing around with math. He told me that it was important to practice your craft every single day so that when the muse hits you, you don't waste it relearning the basics. He went on to say, it isn't even adequate to practice the same techniques - you needed to take this "down time" to acquire new ones.

I think of my time that way. Preparing for when the muse hits me or when the need to be practical forces the issue. I don't know when, if ever, this knowledge will come in handy but I spend my "down time" learning it just in case. In a number of instances, I have learned a lot more by "breaking the rules" than by following them. I understand well that my philosophy doesn't apply to most people or even to myself in all situations.

I try not to speak in absolutes such as "Never do X". Unfortunately - applying common sense and critical thinking is not universal. Otherwise, we could just say "Never do X unless there is a good reason". I just saw a good opportunity to expand and enhance:

If you're only playing around and throw away the code after you've finished, that's fine -- but ignoring all of the knowledge available about what works, what doesn't, and which ideas seemed good but turned out to have problems means you're shortchanging yourself (and anyone who relies on the code) out of false hubris and false laziness.

Cheers - L~R

I read an article some time ago (I don't remember where), about a class for rock climbing where they used exactly the opposite approach. They would start new climbers off trying to climb a relatively simple wall.

After the students had a little time falling and struggling, the instructors would go back and explain the basics to a group who were much more focused on learning not to make the mistakes they had just experienced.

I think we can all agree that random practice without comparison against a standard or correction by a more experienced mentor won't be very effective.

On the other hand, the only way to learn how to attack a problem from scratch is to do just that. It sounds like the OP has a strategy that makes sense for his learning style. It also happens to be one that has served me well in my career.

You are making a distinction between research and production code. I thought the OP made it clear he was talking about the former.

G. Wade

Ok, all fair enough and a good challenge to me. However, you keep talking about webservers and the relevant standards or RFCs. In my case, though, what was I supposed to read? The only standards I can think of are things line the Unix or Linux file system hierarchy standard, and that's not relevant to my concern. (At least, if it is, the connection isn't obvious to me.)

I ended up doing this because I was rereading Intermediate Perl, and I kept thinking about the general problem that file system crawling presents: What do you do when (1) you need to drill through a structure, (2) you have no way to predict in advance how far down it goes, and (3) at each branch the items you find may be a simple thing (a file) or a complex thing (a directory containing zero or more directories and zero or more files)? How do you build a map of such a structure in code? Once you have the map, how do you reorganize it for printing? Perhaps you don't want to print it, but you want to extract one piece of information (the byte count) about one of the types of thing in your map (the files). How do you do that most efficiently? The majority of what I do involves files and folders: scanning them for specific types of things, checking their size, updating them, etc. So it's a problem I need to care about.

If I understand you correctly, you're saying that my time would have been better spent reading through the code in File::Find. Is that your point? If not, then I would be curious to know what you would recommend I do. But, please, no more webservers. I'm not writing one. I promise.

In my case, though, what was I supposed to read?

Fair enough. In your case, a good POSIX or Unix book which describes how filesystems work. In particular, some of the grottiest code in File::Find detects such weirdnesses as recursive symbolic and hard links that can turn a nice tree structure into a complex graph with cyclic relationships. That's a common enough situation that will not be obvious to most autodidacts trying to deal with this sort of problem.

How about dealing with orphaned symlinks? Non-regular files? Mount points? Overlays?

It's also a very common problem in graph traversal, of which "How do I find files under this directory according to some pattern?" is a refinement.

I kept thinking about the general problem that file system crawling presents....

Of course. This is a good example and learning opportunity. By no means do I mean to suggest otherwise. Yet a robust solution to the problem is far more complex than you might suspect.

Perhaps I'm reacting poorly to the phrase "reinventing the wheel", which connotes to me that you've examined the existing solutions and found them lacking in some means. (In my experience, the people who claim that they have really haven't, not in any detail.) Perhaps I've misunderstood you.

If I understand you correctly, you're saying that my time would have been better spent reading through the code in File::Find. Is that your point?

In specific, I can't recommend reading that code. It's not great. It does, however, address a lot of problems that occur in the real world that a robust filesystem walker does need to address. What's a good way of agreeing to the general question but backing off from the specifics?

Perhaps I'm reacting poorly to the phrase "reinventing the wheel", which connotes to me that you've examined the existing solutions and found them lacking in some means. (In my experience, the people who claim that they have really haven't, not in any detail.) Perhaps I've misunderstood you.

I didn't mean "reinventing the wheel" that way, but given how loaded the phrase is, I suppose I knew I was asking for trouble. To clarify: I never intended to replace File::Find. I wanted to see if I understood some of what I had learned from Intermediate Perl and the first chapter of Higher Order Perl, and the way I tested my understanding was by selecting a subset of File::Find's functionality and trying to reproduce it - not because I thought I could do better, but as an exercise. My original post (far) overstated how well I succeeded at even that limited task, and I see that the title is misleading. But I'm glad that I provoked strong responses. I was curious to hear what other, more experienced monks thought of this as a learning exercise. As I've said a few times, I'm completely isolated as far as programming goes, so all the feedback has been very helpful.

In any case, I appreciate your responses; the comments about 'prior art' are especially helpful. Although I use plenty of modules for their functionality, I haven't made enough use of the actual source code as a teaching tool.