Communications of the ACM

The business of software

The Chaos Machine

By Phillip G. Armour
Communications of the ACM, January 2016, Vol. 59 No. 1, Pages 36-38
10.1145/2846086
Comments (1)

Modern software projects can be quite complex. There are many interacting variables and many feedback loops where an action effects an action that changes the first action. If the loops provide negative feedback they will tend to correct ineffective behavior and poor results but sometimes we may attempt solutions that actually accelerate the problems. A classic case of this is when a project experiences schedule difficulties and throws additional staff at the problem, which often causes even worse slippage. In many cases, ill-informed attempts to control projects can actually result in considerable turmoil, even chaos.

Back to Top

The Chaos Machine

At a workshop on the management of modern technology projects to an audience of about 20 telecomm project managers, I addressed how projects and management have changed over the years. By way of a metaphor, I unveiled the "Chaos Machine."

I set the machine in motion and watched the audience's reaction. They were clearly quite surprised at how it worked because how it worked was, well, quite surprising. I stepped around the table on which the machine stood and turned to the audience to make some points about what was happening. A sudden noise behind me startled the workshop participants. "Did you see that?" One of them remarked, "... the darn thing just hopped. All by itself."

To call it a "Machine" might be a little strong since it was really just a pendulum system. Into a heavy wooden base were embedded two sturdy vertical metal supports. Across the top of the supports was a taut wire. From this wire, and tightly bonded to it, were three pendulums of equal length evenly spaced along the wire. Each pendulum was a heavy builder's plumb bob. Each of the metal supports had clips that allowed me to take one or two of the pendulums out of the system.

Back to Top

Simple and Predictable

The first demonstration used the simplest setup of the Chaos Machine (see Figure 1).^a as a metaphor for the simple projects of yesteryear. Clipping the outside two pendulums to the supports left only the middle one free; I pulled the remaining pendulum a short distance perpendicular to the horizontal wire and asked the participants: "What will happen when I let this go?" The audience response was quick and confident: everyone agreed the pendulum will swing back and forth at a regular rate.

This system, like projects of long ago, is simple and very predictable. The rate of oscillation is a function of the length of the pendulum and the amplitude dependent on the initial displacement modified by normal friction. Given the initial conditions, the behavior of the system at any point in time is predictable; so much so that we can, and do, create clocks out of similar systems.

Even easy projects are not as simple as a pendulum. But projects with few interacting variables tend to be well controlled, predictable and, well, easy.

Back to Top

Complex and Predictable

The next step was to take out the center pendulum and put the other two back into play (see Figure 2). I asked the audience: "If I let these two pendulums go at the same time, what will happen?" The audience was less sure, so I pulled them back and let them go.

The behavior with two pendulums is interesting. The periodicity of each pendulum is still a function of their length. But there is also an exchange of energy through the twisting of the horizontal wire and it affects the pendulums. If one plumb bob swings a lot, it twists the horizontal wire. This transfers energy to the other pendulum, which causes it to swing across an increasingly wider arc. What the audience saw was that one pendulum would slow rapidly, as if a brake were being applied, until it stopped completely while the other would speed up equally dramatically. Then the energy would be transferred back to the first pendulum and the wildly swinging pendulum would slow down while the one that had stopped would speed up until it was swinging across a wide arc and the other one would be motionless.

Conservation of momentum applies: when one pendulum was stationary the other would be at its maximum displacement. This system has two periodicities: the swinging of the pendulums and the transfer of energy across the horizontal wire. The rate of the second is a function of the twisting moment of the wire and is somewhat independent of the periodicity of the swinging pendulums. This is a more complex system but, like the simple pendulum, it is also largely predictable—from a known starting condition a future state can be computed with some effort.

---

When projects get more complex, our simple models no longer work well.

---

When projects get more complex, our simple models (think: the waterfall development cycle) no longer work well. Different and more complex project variables interact and change how the project will behave.

Back to Top

Complex and Unpredictable

In the workshop, we moved on to a discussion of the challenges of modern technology projects with all their interacting elements and it was time to fire up the most complex level of the Chaos Machine (see Figure 3). With all three pendulums free, I asked the audience: "What will happen now?" The participants did not have an answer.

This system is example of the "Three Body Problem." It has been the source of much thought and calculation since the time of Isaac Newton and the French mathematician Henri Poincaré was awarded a prize from the King of Sweden for his work on it. When a system has three or more interacting variables it is possible and even likely the system will be intrinsically unpredictable—even knowing everything about the system a priori, it may be impossible to predict its future performance.

I let the pendulums go. Sometimes two of them would stop and one would move, then the stopped pendulums would start up and the moving one slow down. Their behavior is quite arresting. I let the pendulums do their thing and turned back to the audience.

This was when the system "hopped." It hit a chaos point where the interaction of the pendulums became turbulent and some 20 pounds weight of base, supports, and pendulums hopped across the table. The pendulums crashed together and the system stopped.

Back to Top

More than Three Variables

This demonstration was intended to illustrate a point about modern projects and how managers should view them. With a significant number of interactive variables (as in three or more), it is possible, even likely, that an unpredictable system can result; that is, the outcome will often not be predictable at the beginning of the project. It is also possible a chaotic system can develop, that at some point the interaction of parts of the project can become unstable, they can "crash together" and whole thing could grind to a halt.

Modern projects have a lot more than three interacting variables so they could behave in ways that are challenging, puzzling, and unforeseen. On the plus side and unlike a pendulum system, projects can be self-monitoring and self-modifying. So with the right measurements in place, they should be able to detect a potential lapse into chaotic behavior and take corrective action before it does.

Back to Top

Planners

At this workshop, I was talking to a group with a reputation for being highly planning focused. They assumed that, if a project was intensely and minutely planned, everything would naturally and inevitably proceed in an orderly and deterministic way. As project planners, they further presumed any variance from the plan must be a result of inadequate or incomplete planning; when confronted with variation in their projects they tended to try to plan their way to success. And when projects did go off the rails, their response was to stop everything and replan. Sometimes that works, but sometimes it does not and managers of today's projects must have more flexible approaches to managing changes in their projects.^b Modern agile project management approaches have been developed to deal with these complex and unpredictable situations. But there are still plenty of by-the-book project planners out there who are surprised and sometimes baffled by the way their projects behave.

Back to Top

The Chaos Point Redux

I did not see the Chaos Machine crash. But it certainly surprised the audience (and me) and made an interesting and memorable point that gave us some talking points in the rest of the workshop.

I retired the Chaos Machine to my basement and sometimes, when working on something else, I would set the pendulums a'swinging and wait hoping to see them hit the chaos point again. But they never did. So perhaps this holds out some hope: it is very clear that modern projects in the business of software are relatively unpredictable compared to projects of the past and require more attention and dynamic and intelligent responses. But chaos and crashing is not inevitable.

Still, maybe I will pop down to the basement and give the Chaos Machine another shot, just in case.

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2016 ACM, Inc.