* Evolution as a concept has been borrowed from biology into other fields with bad results before. The notion of "cultural evolution" in anthropology led to 800 tons of bullshit based on the idea that human history is a continuous evolution from "primitive" cultures to "advanced" cultures. Sure, we can say that Black and Decker makes better drill bits than Upper Paleolithic tool makers, but does McDonald's make better food? Does Pat Robertson make better religion? Does Smith and Wesson make better weapons (judged societally rather than techncally)? Does Brittany Spears make better poetry? And is a so-called "primitive" or "stone age" culture like the Australian Aboriginies the result of 40,000 years without cultural evolution or of 40,000 years of continuous change in areas that are not measurable by our current methods?

* I think the distinction you make between individual and population is critical for the concept of evolution but I'm not sure what constitutes "an individual" and "a population" in the software world. Suppose an individual is a concrete instance of an abstract pattern. Individual change would be what happens when the instance is run, population change would be what happens over the course of changes to the software that produces the instance.

* Or suppose "population" is the sum of all thoughts of all programmers and "individual" is one programmer's thoughts. The process by which an idea like object-oriented-programming enters the population is related to how it enters the code of individual programmers but is a bigger question. The fact that an individual godzilla can still use perl4 doesn't say much about how the rest or our code has changed since perl5. (Maybe we should abandon the notion of software evolution in favor of software creationism - the gods gave us perl5 and it was good. :-)

* I like your analysis of Word as a better reproducer, but there are other ways of measuring the survival and reproduction of ideas. Word processors as a population adapted to the changing software ecosystem with the popularity of the web by including things like hypertext linking, so did Word as an individual.

* Metaphors, are by definition metaphors. They highlight certain features of similarity between things and background others. When we say "that surgeon is a butcher", we don't mean he sells human body parts for food, we mean he has no fine-control in his cutting methods. To think that every feature of biological evolution will map onto every feature of software evolution is not helpful. What's helpful is pointing out the places they diverge and converge. In that sense, your posting is a necessary thing to make the use of the word helpful. So thanks!

Biological evolution takes time, and lots of it. Even artificial selective breeding programs generally take a lot of time for relatively small gains.

Only because the time between generations could be lengthy. A selective breeding of insects could go relatively quickly, since each generation will only take a few months, weeks, or even days.

Some of the genetic algorithms (a super search for that should bring some interesting nodes) could take a few milliseconds to go through a single generation. So in this case, the breeding program could take no more than a few minutes of CPU time.

I do agree, though, that this "evolutionary" design isn't necessarily going to produce the best designs. In the example of genetic algorithms, it could have been that a combination which would have produced a great solution in a few more generations, but was reaped away because it didn't solve the problem very well right now. Not only that, but since this is all happening so fast that its difficult to analyze each generation by a human, it is likely the programmer will never see the potential that solution had. If anything like natural selection can be applied to software design, it must be a conscious effort on the part of the programmer.

That said, I agree that few small pieces of software "evolve" in any meaningful way. However large ones often do, for exactly the reasons that you describe near the end. As an example, read this description by Linus Torvalds of what Linux development looks like.

And a design note. A large fraction of people's misunderstandings about evolution come down to their trying to say that evolution should be good or right. It isn't. It is about what actually happens and is effective. It doesn't matter whether we want to see evolution happen, evolution is what is going to happen, ideas that can reproduce themselves will become common. You can argue that it shouldn't, you can fight it, you can cry about it, you can accept it. Doesn't matter. It happens anyways.

For a classic essay on this, read Lisp: Good News, Bad News, How to Win Big, which gave us the phrase, Worse is Bettter. (The words "worse" and "better" come from describing the same thing by two different sets of norms.)

I like your post, if for no other reason than the fact that you are one of the few people who throws around the term "evolution" and actually understand what biologists mean.

Well, I do actually have a background in both evolutionary theory as well as the philosphy of biology.

That said, I agree that few small pieces of software "evolve" in any meaningful way. However large ones often do, for exactly the reasons that you describe near the end. As an example, read this description by Linus Torvalds of what Linux development looks like.

Yes, and I guess what I am suggesting is that evolution is something that may be happening later in the life cycle of software, but it doesn't give us useful insights into the design and early development stages.

For me the term evolution in the linked Linus Torvalds letter is used in the broader sense. That one that simonm has reminded us.

The sense in which program development is similar to evolution is somewhere between the level of detail you describe here and the very general "things change". The driving imperative in evolution is "adapt or perish". Of course, that's true of anything geared to a marketplace, and evolutionary design might just as well be termed "market-driven design".

According to this article on the topic, "evolutionary design" is meaningful as a contrast to "planned design". In the latter, you design, then you code to spec. In the former, both are ongoing processes. If you object to the use of "evolutionary", you might prefer Just-In-Time programming for a similar concept: programs are written and changed in response to demand, rather than in far-sighted anticipation of it.

Sure, in the context of modern biology, evolution means something very specific: population dynamics based on genetic inheritance with mutation and selection. But step outside of the field and the rules start to blur. For example, you can look at changes in languages as a type of cultural evolution, with populations, inheritance, mutation, and selection -- even though there's no physical "genetic material" being exchanged.

More generally, the word "evolution" still has a raft of meanings that predate modern biology. Miriam-Webster gives us a sense:

1: one of a set of prescribed movements
2 a: a process of change in a certain direction : UNFOLDING b: the action or an instance of forming and giving something off : EMISSION c (1) : a process of continuous change from a lower, simpler, or worse to a higher, more complex, or better state : GROWTH (2) : a process of gradual and relatively peaceful social, political, and economic advance d: something evolved
3: the process of working out or developing
4 a: the historical development of a biological group (as a race or species) : PHYLOGENY b: a theory that the various types of animals and plants have their origin in other preexisting types and that the distinguishable differences are due to modifications in successive generations
5: the extraction of a mathematical root
6: a process in which the whole universe is a progression of interrelated phenomena

Given that we already have a term for software development that's specifically intended to echo biological evolution, "genetic programming" or "genetic algorithms," I don't see what we would gain from restricting the use of the word evolution in discussions of software development.

While I understand the point the OP is trying to make, I think it's a tempest in a teapot, because the word "evolution" has a broader range of meanings than that indicated by the phrase "biological evolution."

I realize "evolution" has many and varied meanings, the definition I gave as the lead in quote says as much. But, if we are just using "evolution" in the broader generic sense, why aren't these new methodologies labelled like: "The Changing Method of Software Design", or "The Working Out Approach to Software Design", or "The Unfolding Design Methodolgoy" or something similar?

I'll tell you why, it is precisely because of the biological connotations of the term 'evolution', including both the apparent scientific legitimacy of the theory, and the seemingly successful designs seen all around us as a result of biological evolution. And, the wrong, but all too common, notion that biological evolution is progressive.

... why aren't these new methodologies labelled ... "The Working Out Approach to Software Design"...?

Poor marketing; that sounds too much like a health and fitness book, perhaps featuring chapters on "interface calisthenics", "bench press your test suite", "debugging endurance", and so forth.

... "The Unfolding Design Methodology" ...

That one at least sounds lyrical.

Again, I understand the point you're making, but I think you're painting with an overly broad brush.

Anonymonk,

A very interesting and impassioned post, much of which I agree with, however I gently take issue with a couple of points:

• It is entirely possible to develop and 'grow' algorithms through a process which models evolution through natural selection. This has been demonstrated albeit at a simple level here in the monastery. A search should turn up the relevant threads. I know of no actual production code which has been developed using this method, so calling it a development methodology is a tad premature.
• It could be argued that any development methodology where independent teams develop similar modules of code in parallel (population) and then compare their code (variation) against predetermined criteria before reworking the code in the light of the lessons learned (evolution) bears a likeness to the biological process, the major difference being that the fitness criteria are not determiined by the environment but by the observer.
• I would disagree slightly with your point about evolution merely producing better reproducers: This is the generalised result of the types of evolutionary pressure found in nature, however if the criteria used to judge the fitness of an algorith or module are correctly chosen, this does not necessarily need to be the result. As a further point, I would add that Word is the dominant member of the population because of the pressures of the market, which have been manipulated, quite apart from the software development process.

Though provoking, none the less and I wholeheartedly agree with your crusade against the marketroid crud which can be used to sell software.

It is entirely possible to develop and 'grow' algorithms through a process which models evolution through natural selection. This has been demonstrated albeit at a simple level here in the monastery. A search should turn up the relevant threads. I know of no actual production code which has been developed using this method, so calling it a development methodology is a tad premature.

I totally neglected, though intended, to bring up evolutionary programming in the sense of genetic algorithms and the like. These do try to mimic important aspects of biological evolution and are, IMHO, quite properly associated with biological evolutionary concepts. Note, read some papers in genetic algorithms and methodology and the researchers nearly always discuss the particular mechanisms they are trying to model, and how their models differ from models in evolutionary biology.

But no one in those fields is even suggesting that genetic algorithms could be usefully applied in the design and implementation of even a simple text editor. At least not at the present state of the field.

I was only referring to the use of evolutionary principles in the larger software design and development methodologies. That is an important clarification, thanks!

Not all the human plans make sense, but we will never stop thinking and trying. It is that 0.1% hope that attracts us, and actually allows us to be successful.

I fail to see what my name has to do with the issue at hand. If this were posted by one of the monks on the Worst Nodes of all time, or conversely, on the Best Nodes of all time, would that have anything at all to do with the content of this post? More interestingly, would it make any difference in the way you form an opinion of this same content?

If you have done work previously and shared with us, I can see the article in the light with your other work. I per say, dont' build opinion about the content on the merit of the user, but it helps me to see the perspective.

It also helps me to track the node and build better association about the content. If you were to comment further about this over here or in some other thread, I can see continuation easily.

Geesh, I'm pretty happy when other Anonymonks post good stuff. That way, you might confuse my cruft with their, well, good cruft.