This sounds all to familiar to me. Been there, done that way too many times. Unfortunately, I've also worked on projects where extensive and comprehensive analysis and design was done; comprehensive (and usually huge) specification documents were drawn up, test plans written and projects plans laid out. And still the design had to change, the code be re-written etc etc.

That's why (for me) the most exciting (and don't use that word often or lightly) thing I have seen, and the only new design 'philosophy'--'paradigm', 'methodology', call it what you will--that has registered anywhere above "Oh god, here where go again" in the last 20 years, is Extreme Programming.

Much of it is in part, familiar territory for me in as much as I have, through choice or dictum, used one or more of the XP techniques on one or another project down the years. I'd even go as far as saying that I had realised the benefits of some of them a long time since, especially small teams/pairs, iterative (RAD) development, even the practice of defining the test criteria and writing the testcases before the code, but the simplicity and clarity with which the XP picture brings all of this and many other good practices (not ideas) into a single, operable and cohesive whole is--I pondered for at least 5 minutes for the right adjective here--elegantly concise and simple. I guess I should have said extreme, but that has connotations that I don't think sit well with management types.

I can't help but feel that evolving specifications for projects lead only to code that is harder to maintain, more of a "kludge" that does the job than a carefully crafted entity.

The upshot is, that if the project plan is designed around the concept of continuous improvement, refactoring (latest buzzword), getting the simplest possible function set working and testing correctly as early in the project as possible, and then extend and refine functionality as you go, the process of incorporating evolutions in, and backing them out becomes a simple part of the daily process.

What I used to call, 'suck it and see' programming before they invented fancy buzzwords to describe the process.

Of course, this is all just so much hot air if your unlucky enough to be in one of those shops where dogma and rhetorical programming practices are the norm.