This continues to be a most interesting episodic series of articles.   Thank you for continuing to post them.

This is also one of the first “articles on Agile” that tacitly acknowledges some of its deepest flaws, and that offers up a possibility for correcting them.   While I generally agree with the stratagems and professional perspectives you have listed here, and so I am not rebutting them, I still come away with the feeling (and it has grown over the many years now), that the much-maligned “Waterfall” was probably, in a great many ways, right!   The construction of a great many tall and heavy things follows this, usually by-law.   The construction of things that don’t move, and especially, the construction of things (such as software and other big, dangerous machines) that do.

There is definitely a political problem with software:   for a long time, the team doesn’t appear to be visibly doing something, and management naturally wants to see “code writers” “writing code.”   Nails being hammered into boards.   Concrete being poured.   (And they’d better not see a jackhammer being taken to that new slab ...)

But in reality ... and this goes back to the Managing the Mechanism e-book (Kindle, iOS) ... software is actually a machine with near-infinite internal complexity.   Everything is, or can be, “coupled” to everything else.   Yes, you can have committee meetings about what color a particular room screen should be painted, but not much more.   Analogs to building-construction also fall flat:   buildings don’t move.

I also find that it is important to separate coding from testing.   A lot of development may be done before it is possible to do more than superficial testing, because the moving-parts that are to be moving-together must of course first be built.   The overall mechanism must be broken down into sub-systems and these must be tested:   both unit-level and integration testing must proceed continuously.   You cannot do this unless the completed-and-tested parts do integrate with one another, and will integrate with what is yet to come, all without ever breaking.   You can build things in-parallel, but you absolutely can’t design them that way.  Having separate design teams forces the two teams to communicate, which forces the design itself to be in a communicate-capable form.

And this one key point inevitably becomes clear:   there must be a master blueprint, a master design document that is built first(!) and that does not(!) change.   You can sneak-around that requirement if you want to, but the teams ... no matter how many teams there are or aren’t ... do not “snap into rhythm and start pulling together as one” until that over-arching, master design is clear to all and stable.   Anarchy is no substitute for design.

“I See Dead Projects,™” and most project failures or near-failures come from process problems only.

Because of the near-infinite complexity of the thing, computer software suffers from “the Jell-O® Problem.”   When you so much as touch it, anywhere, it wiggles everywhere.   If the design of the entire thing is not pretty-much known in advance, and adhered to, the situation becomes even worse:   you are now ripping parts out, after the design changed on-the-fly (or, was arrived-at on the fly), and it is basically impossible to know where the dependencies and repercussions will be.   I personally think that Agile, as practiced, makes the problem much worse for two reasons:   (1) it encourages practices that rob the structure of its internal stability, and (2) it sets the expectation, both in the management and in the team(s), that this is “the right thing to do.”

We see an even-more insidious manifestation of this problem today in mobile applications that are being built using rapid-development frameworks such as Ionic or Phonegap.   Apps are being built “rapidly,” mostly by amateurs, and, even though the amount of code that the team has written is comparatively small, the source-code base of which the application actually consists might be > 120,000 lines.   There is a frightening level of internal complexity there, but superficially the presence and the implication of that complexity is not recognized.   Change is made like the (bad) cell-phone commercial:   “can you hear me do you like it now?”   After a surprisingly-few iterations of this, the entire application is a brittle pile of broken glass.   Not a “tool problem.”   A “process problem,” but one that the tool encourages.

I like this article, nevertheless, because you deal with the realities vs. the theory or the “principles.”   What is actually done in, and by, a seasoned and hard-working professional team.   Versus what the manifestos say.   Again, thanks for sharing.