I daresay though that Perl might not be the place for you to start, depending on what you want to learn. Like you, I started out in C, stuck with it a bit, and then moved on to Java before I started to touch Perl, Python, PHP, &c. It provides you with a solid foundation in the way memory management works, and if nothing else will lead you to respect what Perl and its cousins can do more.

So by all means, reinvent the wheel when you have the luxury of doing so. It's a great way to learn the trade. The principle of NOT reinventing the wheel applies mostly when you're more concerned with *getting* something to work than with understanding the process of how it works.

Cheers!

Thanks for the encouragement. I suppose I should have said that not reinventing the wheel makes perfect sense for knowledgeable programmers who are trying to get work done in real time. But I also see this advice all the time given to beginners (in beginner's books, in tutorials for newbies, etc.). That's the context I had in mind above.

As for other languages, I was planning to work through SICP this summer. From what I can tell, it is an outstanding introduction to programming full stop.

There are several things here I see as worthy of discussion, or at least mention.

First off, in the pursuit of learning go with whatever floats your boat. As I'm sure you are well aware different modes of learning suit different people. If you can identify what works for you, you are way ahead of the rest of the crowd.

Second, why reinvent wheels? I'm not saying you shouldn't reinvent wheels for the purpose of learning, just wondering if there isn't some other sort of tool or machine you could have had a swing at? You imply ('writing the same ... things over') that there is scope to put some aspects of your code into modules. That may not have needed recursion, but could well have led you to modularised code and OO which are much more often used for fun and profit in any case. What I'm suggesting is that instead of reinventing a wheel as a focus for learning, you may have been able to tackle something with an immediate pay off that is specific to your application area.

Third, you chose to go the 'clean room' route - you didn't look at the construction of the existing wheel before you started out on your own implementation. That's no bad thing but, particularly with software, it's pretty hard to judge how good your 'answer' is just by comparing it with someone else's answer. I'd suggest for completion of this exercise you should post the code to Cool Uses For Perl and ask for some criticism. I'd recommend that you link this thread in your CUFP post to provide some context just in case the 'reinventing the wheel' police come gunning for you.

Last, as I'm sure you know, teaching others is a great way to learn. Poke around Seekers of Perl Wisdom, offer a little help where you can, read the sage (and otherwise) answers of others, and your Perl will progress at a great rate.

So, you're saying the problem is that using CPAN modules makes all of your work so trivial and easy that you're bored? You should do marketing for Perl!

Seriously, the idea is that it makes the totally generic stuff (like finding .ppt files) get out of your way so you can take on more in your specific problem domain. If I had to write my own argument parser every time instead of using Getopt::Long and friends, I'd never have time to write a significant program.

I would also say I've learned a lot about writing good APIs, documentation, and code, from using CPAN modules.

So, my advice is to think bigger. If your actual work is too easy, make it do more than it has to. There has to be something more useful to spend your coding time on than rewriting File::Find.

So your itch isn't writing argument parsers. That's fair. But does that mean others shouldn't scratch their own itch by writing argument parsers? Have you forgotten it was mis-guided souls re-inventing argument parsing who created Getopt::Long and friends in the first place? How much bit rot will those same modules suffer from when the rest of the world "sees the light" and stops re-inventing their own argument parsers?

Seriously, if I had to write my own Perl 6 parser every time instead of using Perl 5 and friends, I'd never have time to write a significant program. ;)

We've all got our itches to scratch.

But does that mean others shouldn't scratch their own itch by writing argument parsers?

Yes. At least, it should.

One of the worst problems in software development is that many programmers are incapable of or unwilling to read existing code. It's too hard. They don't like the indentation. It's not fun. It's messy.

Whatever.

Matt's Script Archive reinvented a lot of wheels. It's difficult to estimate how many spammers still ply their trade thanks to his code -- written by hand -- ignorant of good security techniques and Internet standards. I have a silly habit of reading hand-rolled CGI argument parsers. Almost every one I've ever written make the same five mistakes (a couple of them active security holes), and most have at least seven bugs I can identify within seconds of reading them.

Writing code for didactic purposes may be appropriate, if you learn something from it. Did you seek code review? Exploration testing? Adherence to the relevant standards? Algorithmic analysis? Did you compare the results to results from a widely-used, well-tested alternative?

Now perhaps you are the kind of coder who researches the problem domain, writes a comprehensive test suite, and validates the results against a well-known and well-understood set of specifications and requirements. If so, good for you.

Yet the person who believes that he absolutely must, for example, write his own web server for production purposes without reading and understanding the relevant RFCs or without consulting the current art of the domain for the purpose of writing something nebulously "better" or "faster" or "easier to understand" or "just so I can learn how to use hashes effectively" exhibits unprofessional, immature, irresponsible, and actively harmful behavior.

Have you forgotten it was mis-guided souls re-inventing argument parsing who created Getopt::Long and friends in the first place?

Feel free to respond with the tired old saw about how Einstein was a mere patent clerk when he developed his theory of relativity, as if that proves that an interested amateur outsider can significantly advance the state of the art by breaking free of existing conventions. It's an urban legend, for one -- a particularly well debunked one. Yet even if it were true, it proves only that it's possible perhaps once a century, and certainly not that it's likely.

(I rather suspect that people who've written and maintained argument parsers -- especially if they've dealt with feature requests and bug reports -- have a much better idea what argument parsers need to do than someone who's never written an argument parser before.)

Well, if you've already tried Getopt::Long and found it lacking, and you're working on a personal project where others won't have to maintain your code, and you aren't charging me by the hour for it, then sure write your own args parser. But don't start there and check Getopt::Long later. There's an opportunity cost to writing redundant code when you could have been doing something new and valuable.

Thanks to everyone for feedback. It looks easier to respond in one place, so here it is.

@ Zentara: I agree that there has to be some limit where you stop trying to do it all yourself. And there's no danger of me ending up in Assembly: Even the little bit of C that I've done was enough to make me (as Lawliet and starX say) very, very grateful for Perl.

@ Perrin: I wouldn't quite say that I'm bored. Let's all agree not to put it that way to my employer. Still, your advice is good. I have tried to "do more than I have to" with the code I write. (For example, I doubt that many 100 line utility scripts provide command-line parsing (via Getopt::Long) and a full man page (via Pod::Usage).) I actually do that for the same reason I tried this "reinventing the wheel" experiment: to push myself, to learn something new. Nevertheless, you may be right that my time would be better spent on new work.

@ Chromatic: I hear what you're saying, but much of what you wrote doesn't straightforwardly apply to me. (I admit my case is unusual.) For example you write:

Writing code for didactic purposes may be appropriate, if you learn something from it. Did you seek code review? Exploration testing? Adherence to the relevant standards? Algorithmic analysis? Did you compare the results to results from a widely-used, well-tested alternative?

As I said in my original post: I am the only person writing code in any sense where I work. And quite frankly I'm a complete amateur (and novice). It sounds like you are primarily thinking of professional, full-time programmers. (A reasonable enough focus for you, but it's not who I am.) Still, to respond to your questions:

• Nope, no code review. I don't really have anyone to ask. See below for why I'm not going to ask here as far as this experiment goes.
• I have no idea what exploration testing means in this context. So, I guess I didn't do it. I'm only just learning how to write tests at all. Yeah, yeah, I know, get out the torches and the pitchforks. (Coincidentally, I bought your book on testing about a week ago because I figured it was high time I learned.)
• There are standards for filesystem traversal? Cool. Who knew? Nope, I didn't look at those either.
• I do have an inkling of what algorithmic analysis means in this context, but nope, I didn't do that either. Again, I have no training or background which would allow me to analyze an algorithm in any very substantive way. I thought about what I needed to do, tried things, looked at how they failed, tried some more things, and so on. However, I probably should have done some algorithmic analysis. When I run my code against a fuller directory (eg, my $HOME), it wets the bed with a whole lot of "Deep recursion on subroutine main::walker" errors. During development, I only ran it on a smaller sub-directory. Damn.
• And, finally, I am in the process of comparing my results and code with File::Find. I certainly appreciate File::Find more than I did two weeks ago.

@ Grandfather: I have one module written for my work code, and you're right that I should continue to work on that. I understand OO Perl well enough to use other people's OO modules comfortably, but for my own code, I find procedural (with a little bit of functional) more natural so far. I should probably work on that too.

I wholeheartedly agree that teaching is a good way to learn, and I do actually answer questions on the rare occasion here, at Stackoverflow and on the Perl Beginners mail list. It's a great way for me to solidify my understanding of things. I also read a lot of posts to see how people approach things. I think I will pass on the invitation to post my broken wheel to the Cool Uses for Perl part of the site. (See above on "Deep recursion" errors. Damn.)

So where am I left? Well, I'm less thrilled with my experiment, now that my wheel doesn't roll properly except on small streets. I also think people poked some good holes in my reasoning. On the other hand, I do feel a lot more comfortable with callbacks and code references. Maybe this wasn't the best way to spend last week, but I'm not yet convinced it was a complete waste of the week either.

It sounds like you are primarily thinking of professional, full-time programmers.

Not at all.

perrin had a pithy quote about opportunity costs. I believe that the goal of programming is to solve problems, not to produce code. If you can solve problems without writing code, so much the better. Not everyone agrees.

I do believe that an effective learning technique is doing (which is one reason I believe the testing notebook works in it current format), but I also believe that unguided doing is a poor use of time. It's trivial to write a simple HTTP server, if you ignore most of the HTTP specification -- but as a learning technique, writing it by sniffing socket traffic from a single web browser on a single operating system is a poor way of understanding how HTTP works. There's no substitute for reading the relevant RFCs.

If you're only playing around and throw away the code after you've finished, that's fine -- but ignoring all of the knowledge available about what works, what doesn't, and which ideas seemed good but turned out to have problems means you're shortchanging yourself (and anyone who relies on the code) out of false hubris and false laziness.

Writing good software is difficult, especially if it interacts with other users and other software. There are no shortcuts besides well-honed good habits. I want to encourage people to develop good habits and discourage them from developing bad habits.

I believe that the goal of programming is to solve problems, not to produce code. If you can solve problems without writing code, so much the better.

The single biggest contribution to reliability, security and maintainability, is using less code to solve the problem.

Note: using, not writing. You may succeed in writing less code by using a one or more modules, but if the modules are over-engineered, too generic, grossly hierarchical, or otherwise verbose, then you can still end up with more code than you need. And that's bad all round.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

chromatic,
If I had to come up with a number, I would wager that 75% of the code I write is "throw away" code. I do not work as a professional programmer and think of code as my creative outlet. It is easy to see from a few links that my primary pursuit is knowledge. As an artist, it is often important to me not to be influenced by existing solutions when solving a problem.

Here is where I make a clear distinction and was happy that you wrote this node. I very seldom have the hubris to believe I have created something superior. I go back and examine very closely existing wheels. I run my code through their test suites. I discover what bad assumptions I made and how others addressed them. I spend way more time on a problem than is required to solve it so I might squeeze the very essence out of it and learn as much as it has to offer.

This isn't the only way that I "waste" my time learning. I spend a lot of time answering questions here and on #perl IRC. I see the common elements of problems over and over again. I see how others responding solve the problem. I learn from what they have to offer and often will summarize the differences in the solutions and why one may be more advantageous over the others. It has been my experience that well worn wheels are able to accomplish the job for 98% of the general problems - there is still 2% that require going beyond existing wheels.

I spend this 75% of the time doing "pure" research so that I can do my best the 25% of writing "applied" solutions. I do make money from programming (hobby vocation) and I do solve real world problems (for myself, for my employer and for complete strangers). This includes very large complex applications and not just 50 line scripts.

After nearly 7 years of programming in perl, I think I would still be a novice if I only wrote code when I needed to.

Note: All the numbers in this node are made up WAGs

Cheers - L~R

Ok, all fair enough and a good challenge to me. However, you keep talking about webservers and the relevant standards or RFCs. In my case, though, what was I supposed to read? The only standards I can think of are things line the Unix or Linux file system hierarchy standard, and that's not relevant to my concern. (At least, if it is, the connection isn't obvious to me.)

I ended up doing this because I was rereading Intermediate Perl, and I kept thinking about the general problem that file system crawling presents: What do you do when (1) you need to drill through a structure, (2) you have no way to predict in advance how far down it goes, and (3) at each branch the items you find may be a simple thing (a file) or a complex thing (a directory containing zero or more directories and zero or more files)? How do you build a map of such a structure in code? Once you have the map, how do you reorganize it for printing? Perhaps you don't want to print it, but you want to extract one piece of information (the byte count) about one of the types of thing in your map (the files). How do you do that most efficiently? The majority of what I do involves files and folders: scanning them for specific types of things, checking their size, updating them, etc. So it's a problem I need to care about.

If I understand you correctly, you're saying that my time would have been better spent reading through the code in File::Find. Is that your point? If not, then I would be curious to know what you would recommend I do. But, please, no more webservers. I'm not writing one. I promise.

To me, the most important point of your post is

And, finally, I am in the process of comparing my results and code with File::Find. I certainly appreciate File::Find more than I did two weeks ago.

Reinventing the wheel by going in a random direction can cause extremely bad habits. Taking your version and comparing it with a working version helps to moderate that effect.

Unlike some of the advice here, I think this is a valid and very effective approach to learning, as long as you don't forget the production wheels that you can learn from. You also have the opportunity to rewrite your code after the comparison to see if you really learned what you believe you learned from File::Find.

As professional programmers, it's easy to lose track of the need to practice. Rewriting code that is not needed for production can help you learn and help keep your basic coding skills from getting rusty.

It also helps to remind us of the sense of accomplishment that we used to get from making something small with our own two hands. When working on big production systems with multiple programmers and man-decades of work, it's hard to remember the simple pleasure of making the computer do something for me, as opposed to keeping the system up and functional.

Good luck in your explorations and don't forget to share your experiences with your fellow monks.

G. Wade

Rewriting code that is not needed for production can help you learn and help keep your basic coding skills from getting rusty.

What are you doing with production code that it does not exercise your basic coding skills?

Well, if not to CUFP, then at least post your wheel, broken or otherwise, in this thread. An important part of the learning process (and perhaps the hardest) is working through the criticism of your answer, but it may be the most rewarding in the long run.

Comparing your code with File::Find won't pick up on stylistic issues or preconceived notions that you may have and that will come back to bite you later on. Many good coding techniques look so natural that you don't even notice them unless they are pointed out. As a trivial example, missing strictures (use strict; use warnings;) won't stand out in your code if you are not accustomed to including them in every script you write. However strictures often make a huge difference to code quality and the ability to identify and eliminate bugs early. If you even just receive one answer that provides a tip on the order of 'always use strictures', posting the code is well worth the dent to your self esteem. ;)

---

True laziness is hard work

No....it's best to happily stand on the shoulders of the giants who preceded us...... and avoid sweating the details unless necessary.

I don't know, I learned a lot from Assembler :)

I wouldn't want to write... well... *snything* in it, mind you, but I'm glad for the experience of having learned it and written some primitive operating systems.

--starX
www.axisoftime.com

should we all be writing Assembly language code?

Well, we should first build our own processors, hard drives, RAM, et al. That way we can really appreciate Perl. ;)

While trying to explain palindromes to my alien friend, he asked "Eks, aeh dneirf neilay motsem-ordni lapnial p xeot gni y rtelihw?"
What can I say? Palindromes are rasemordnilapyasinactahw~

we should first build our own processors, hard drives, RAM, et al.

No thats really a waste of time. I used to be an electronic Tech....thought learning vacuum tubes, transistors, et.al. were important to know.....now all obsolete.

Hard drives, and RAM as we now know it will be obsolete shortly. What you want to learn is the mathematical techniques to model problems.....then you can solve problems whatever level technology or language, you are priviledged to work with.

Programming will change eventually to mainly realtime stream processing, probably on computers run/connected on laser light instead of electrons. Can you imagine a "transistor" that switches light? Then imagine a cpu made out of them.

---

I'm not really a human, but I play one on earth My Petition to the Great Cosmic Conciousness

Scientists who study memory, cognition, and language acquisition would be the first to tell us that there is a huge difference between receptive (listening/reading) and productive (saying) use of a language. People can produce language without being able to understand it (sometimes a side effect of brain trauma or hearing/reading problems). People can understand language without being able to speak it.

As anyone who has thought hard about their experience learning language will realize, we are most skilled in the languages where we have both read and produced the language. To write well in English, one has to read a lot of English. But one also has to write a lot of English. And if we had a good English teacher, we were positively encouraged to reinvent the wheel and try to write our own sonnets or haiku about spring, love, puppies, or anything else that interested us. In fact, the starting point of all good authors is their passion.

We also learned from our best English teachers that good authors also let others read their work - first their teachers and a close circle of friends they trust (or sometimes a remote publisher, because anonymity hurts less). They take in the feedback and improve their writing style, think more about plot lines, listen (finally) to the person that says "don't write a historical novel without research", and so on.

Some of us learned this way to appreciate the passion of others and decided that writing was not our thing. Some of us found writing hard work, but still persisted. But one way or another, we all benefited from the experience, whether as more appreciative readers or as people that grew up into successful professional writers.

Though I haven't seen any studies on this (I admit I haven't looked), my hands-on annectdotal experiences leads me to believe that the best programmers and designers are not that different from the best poets and novelists. To write good code one has to both read and write it. We need to understand a problem in such depth that we get past naive understandings (the equivalent of stereotypes and cliches in novels and poems). We have to love something enough that we are willing to take the time to research the RFC's, learn about file systems and their quirks, and still want to try our hand at it.

It is very hard to understand the complexities of a problem until we have gotten down into the dirt and tried to design and then code a solution. Until we do that, we are viewing fields and mountains from a mile high and see nothing but large patches of green and brown. Eventually, we must take the risk and show that code to others so they can critique our style and plot lines and character development (a.k.a. design).

So long as the would be author (programmer) is willing to both write and read, research and create, I see no problem with "reinventing the wheel".

Your post and the various responses have really made me wonder if we shouldn't refocus our efforts. Instead of slapping people on the hand when they reinvent the wheel, maybe we should focus on teaching them when and how to do so? chromatic makes very good points, but wouldn't they be better taught by encouraging students to write a file system searcher (or other wheel of their choice), and then requiring them to couple the code with a research paper comparing their code with appropriate standards and background reading plus a critique of at least one other program trying to do the same thing? This would, I think, do a lot to more to teach people the skills and judgment to create well designed programs than would all the lectures in the world on "don't".

Best, beth

Reinventing the wheel as a means of getting the job done is usually bad. Reinventing the wheel as a means of learning is good. We've all done that on some level. What else are the practices in books or the tasks we all completed learning something, than reinventing the wheel?

There's one great thing about a reinvented wheel. You can compare your solution with the solution(s) of others and thus learn more than by just writing code or just reading someone else's code. By implementing that wheel yourself, you get a much better understanding of the problem and can better appreciate or critique the decisions made by the other implementor. You can find better ways to write something than what you wrote yourself, you can find that your solution of a subproblem is shorter, but maybe has some drawbacks. You may even find out that your wheel is rounder, better spring-loaded, ... and generally works better. Or maybe not. But it was still well worth the time.

What we need are programmers that will create new wheels to solve the new problems. Most of the old problems are already well solved. Sometimes studying the old solutions, predisposes you to making similar designs....it's ok if they are the best solution, but often we carry mistakes from the past forward into the new designs.

Additionally, do you know how toxic it is to make computer parts, on your own? We should have stuck with the abacus....:-)

---

I'm not really a human, but I play one on earth My Petition to the Great Cosmic Conciousness

• Road map woes
• Necessity is the mother of invention
• Breaking The Rules
• Breaking The Rules II

I would caution you to read those threads closely and not just jump to the conclusion that I think it is a blindly a good idea.