The title of this Meditation comes, of course, from the punch-line of a really bad movie with a classic O’Henry Ending:
But in many ways, it also sums up my career. (Polite pause as the twitter of laughter dies down.) For most of the past 25 years or so, I’ve been involved in projects. Generally, not ones that I had started. Generally, not healthy ones. Dead ones, or very nearly so. My task was to try to “turn them around,” and I generally did. Whether or not my attempts at resuscitation were actually long-term successful, this experience did teach me a lot of the reasons ... and they are human reasons ... why software projects so often go so badly wrong. I’m not going to do any preaching here, although it may seem so. I’m just relating some of my personal experiences in a mortuary project triage. (FYI: Teams-in-place were anywhere from one to fifteen people, most of whom had “split the coop.”)
First of all, these projects typically started out with “a great deal of enthusiasm, but no real plan.” The usual justification was that the project needed to “hurry up to market,” or that the stakeholders in the project “would know it when they saw it” and the managers of the project (if there were any ...) simply gave-up trying to ask them to make up their minds.
And, of course, in several cases, those stakeholders were assured that they didn’t have to make up their minds. “Self-directed teams,” the programmers purred self-confidently, “would produce a ‘potentially viable product(!)’ every two weeks!
“SOP = SOTP.™” Standard Operating Procedure = Seat Of The Pants.
And yet, what happened ... what inevitably happened ... is that everything in the “software mechamism” turned out to be inextricably coupled to everything else. As layers of code were piled on, and as changes pinged-and-ponged throughout all those layers, the whole thing fell down in a heap as the programmers sailed on to the next green pasture.
Many software projects are actually the work of one Guy. (Sorry, ladies ...) That “one guy” might be surrounded by several other people, but this is simply an attempt to scale-up the only modus operandi that this One Guy actually knows: himself. The project “feels its way along” because that’s how he’s used to doing it. (And, because he is a crackerjack programmer, is used to eventually succeeding producing something.) There simply isn’t any experience in being part of a successfully managed project: most programmers, I candidly suspect, haven’t actually seen one. (And there were no Angelic Choirs that started singing when I showed up either, I’m afraid ... no self-sunshine here.)
The underlying reason for these problems, I think, is: a very natural human reaction to what is a virtually-unmanageable technical situation. The objective of the project is to build a self-directing machine ... and to do it perfectly, because nothing less than perfection will do. Viewed as a mechanism, software would be said to have “unlimited degrees-of-freedom.” i.e. “Anything is connected to everything else.” Although the instant-to-instant flow of control within the software is of course described by if/then/else and looping constructs, the actual mechanism is also determined by its internal and external state. This concern for “state” is what causes the coupling. (And it’s also one of the reasons why “Functional Programming” is such a hot research topic.)
My biggest criticism of Scrum, and Agile, and XP, and, well, most “methodologies,” is that they ignore this aspect. They focus, instead, upon the organization and the daily work-activities of the team. They discuss things like “user stories,” which are simply one possible way of trying to express one’s ideas and plans to a customer, but then omit from consideration exactly how that “story” is to become if/then/else, and how that new web of decision-logic is to be tested, and how it both affects and is affected by (“is infinitely coupled to ...”) everything else. As a paradigm, useful in one sense though it may be, it does not and probably cannot (IMHO) go far enough.
“We are building a self-directing machine.” That, quite frankly, is the light-bulb moment that I got from the Managing the Mechanism e-book. It’s something that we can say to business stakeholders, except that it is extremely likely to scare them off. It certainly does, I think, cast some useful insights on what we might be missing in our present-day methodologies. We certainly do need better processes for our work, better ways to describe them, and better ways to inform stakeholders of exactly what we need from them and why.
In closing, one of the most prevalent things that I have seen, in every project that I have tried to turn-around, is disillusionment. On both sides of the aisle. Long before the software had broken down, communication had also broken down, and so had business process (if it ever truly existed). No one builds houses and bridges that way. (For very obvious, flammable and heavy reasons, no one is allowed to ...) I suspect that the seeds of project failure are sown almost as soon as the first plow-blade cuts the soil. This is our problem, as a profession, and we need a better solution to it. Perhaps a different viewpoint is a start.
That’s my Meditation. Borne, as I said, from a most-interesting career path that has not always been a happy one. What do you think? What have your experiences been? For instance, have you worked-through a spectacular success story from one of these other strategies? I’d love to hear it . . . The water in the cooler is ice-cold and there’s beer in the fridge that’s even colder. May the discussions begin?