Communications of the ACM
Doing It Right or Doing It Over?
After a few articles on the publication culture, this one returns to software engineering themes.
I have become interested in agile methods because they are all the rage now in industry and, upon dispassionate examination, they appear to be a pretty amazing mix of good and bad ideas. I am finishing a book that tries to sort out the nuggets from the gravel [1].
An interesting example is the emphasis on developing a system by successive increments covering expanding slices of user functionality. This emphasis on delivering something that can actually be shown — "Are we shipping yet?" — is excellent. It is effective in focusing the work of a team, especially once the foundations of the software have been laid. But does it have to be the only way of working? Does it have to exclude the time-honored engineering practice of building the infrastructure first? After all, when a building gets constructed, it takes many months before any "user functionality" becomes visible.
In a typical exhortation [2], the Poppendiecks argue that:
The "right the first time" approach may work for well-structured problems, but the "try-it, test-it, fix-it" approach is usually the better approach for ill-structured problems.
Very strange. It is precisely ill-structured problems that require deeper analysis before you jump in into wrong architectural decisions which may require complete rework later on. Doing prototypes to try possible solutions can be a great way to evaluate potential solutions, but a prototype is an experiment, something quite different from an increment (an early version of the future system).
One of the problems with the agile literature is that its enthusiastic admonitions to renounce standard software engineering practices are largely based on triumphant anecdotes of successful projects. I am willing to believe all these anecdotes, but they are only anecdotes. In the present case systematic empirical evidence does not seem to support the agile view. Boehm and Turner [3] write:
Experience to date indicates that low-cost refactoring cannot be depended upon as projects scale up.
and
The only sources of empirical data we have encountered come from less-expert early adopters who found that even for small applications the percentage of refactoring and defect-correction effort increases with [the size of requirements].
They do not cite references here, and I am not aware of any empirical study that definitely answers the question. But their argument certainly fits my experience. In software as in engineering of any kind, experimenting with various solutions is good, but it is critical to engage in the appropriate Big Upfront Thinking to avoid starting out with the wrong decisions. Some of the worst project catastrophes I have seen were those in which the customer or manager was demanding to see something that worked right away — "it doesn't matter if it's not the whole thing, just demonstrate a piece of it!" — and criticized the developers who worked on infrastructure that did not produce immediately visible results (in other words, were doing their job of responsible software professionals). The inevitable result: feel-good demos throughout the project, reassured customer, and nothing to deliver at the end because the difficult problems have been left to rot. System shelved and never to be heard of again.
When the basis has been devised right, perhaps with nothing much to show for months, then it becomes critical to insist on regular visible releases. Doing it prematurely is just sloppy engineering.
The problem here is extremism. Software engineering is a difficult balance between conflicting criteria. The agile literature's criticism of teams that spend all their time on design or on foundations and never deliver any usable functionality is unfortunately justified. It does not mean that we have to fall into the other extreme and discard upfront thinking.
In the agile tradition of argument by anecdote, here is an extract from James Surowiecki's "Financial Page" article in this week's New Yorker. It's not about software but about the current Boeing 787 "Dreamliner" debacle:
Determined to get the Dreamliners to customers quickly, Boeing built many of them while still waiting for the Federal Aviation Administration to certify the plane to fly; then it had to go back and retrofit the planes in line with the FAA's requirements. "If the saying is check twice and build once, this was more like build twice and check once", [an industry analyst] said to me. "With all the time and cost pressures, it was an alchemist’s recipe for trouble."
(Actually, the result is "build twice and check twice", or more, since every time you rebuild you must also recheck.) Does that ring a bell?
A good commentary is once again Erich Kästner's ditty (my translation from [5]):
They tell you it’s OK if first you fail;
OK perhaps — but not so practical.
Not all who for India set sail
Hit on America.
References
[1] Bertrand Meyer: Agile! The Good, the Hype and the Ugly, Springer, 2013, to appear.
[2] Mary and Tom Poppendieck: Lean Software Development — An Agile Toolkit, Addison-Wesley, 2003.
[3] Barry W. Boehm and Richard Turner: Balancing Agility with Discipline — A Guide for the Perplexed, Addison-Wesley, 2004. (Second citation slightly abridged.)
[4] James Surowiecki, in the New Yorker, 4 February 2013, available here.
[5] The original reads:
Irrtümer haben ihren Wert;
Jedoch noch hie und da.
Nicht jeder, der nach Indien fährt,
Endeckt Amerika.
No entries found