Communications of the ACM
Communications of the ACM
Extreme Collaboration
Collaborative aerospace design benefits greatly from face-to-face interaction. Colocating design engineers in the same physical environment helps provide all participants immediate information on how requirements might have changed or, conversely, enables them to question or adapt to changing requirements immediately. Being located in the same physical environment enables each of them to approach and interact with whoever at the moment might have relevant information. The group benefits from having multiple members noting changes and catching errors before they risk compromising other aspects of the project. Continual communication among designers is critical to solving problems, discussing alternatives, and questioning assumptions.
Teams collaboratively designing face-to-face, exemplified by war room environments, are beginning to be employed in a range of organizations around the world. In war rooms, team members work together synchronously in all phases of a project, meeting for a variety of purposes beyond status reviews. Studies of war room-based work suggest these environments lead to increased productivityfar beyond what a manager might expect from a traditional design effort. For example, one recent study of co-located programmer teams focusing on software metrics found productivity doubled compared to a company baseline [8]. Similar increased productivity was reported in a study of managers and sales teams in war rooms [3]. Other studies suggest that paired programmers working side-by-side make fewer errors and may also be more productive than their counterparts working alone [9].
Here, I describe a unique kind of war room environment, located in NASA's Jet Propulsion Laboratory (JPL) in Pasadena, CA, where engineers work closely together using a variety of computer technologies, including networked computers and public displays. I call this type of design activity "extreme collaboration" to emphasize working in an enclosed electronic and social environment to maximize communication and information flow. My own recent study there sought to answer a central question about computer-supported cooperative work: How can physical co-location and technology be combined to enable a team to produce a complex space mission design in a remarkably short timewithin a week?
Planning Exploration Missions
JPL provides research support throughout NASA for planning missions in environments ranging from the deep ocean to deep space. In April 1995, a visionary plan established the Advanced Projects Design Team, also known as Team X, to serve as an internal consultant to NASA in designing new space mission proposals, including various Mars probes. Team X sought to improve the speed and quality of mission proposals through a combination of a permanent team of 16 engineering experts from diverse scientific fields and an electronic meeting-room environment. The mission design proposals define all aspects of a mission: how the science requirements should be fulfilled; which telecommunications devices should be used; the amount of power and propulsion that are needed to complete the mission; and which information should be transmitted back to Earth.
Figure. The JPL Team X war room environment.
The 16 Team X engineers provide expertise in the various specialized subsystems needed for space mission design, including power, thermal, and telecommunications. The team leader is also a systems engineer. Most of the engineer-participants have now been members of Team X for at least two years, some as long as five years. The room incorporates public displays, databases of past space mission components and measures, orbit visualizations, configuration graphics, and a publish-subscribe system of networked spreadsheets disseminating information throughout the environment (see the image here).
It is difficult for customers to judge the accuracy of a Team X design proposal when first completed, as it can take years before the related space mission itself is completed, but some verifications based on cost-accuracy models suggest the quality is high. A model designed by a JPL cost group using Team X output predicted the final cost of seven completed space missions to within 5% of the actual result. An independent verification review by an internal NASA customer was within 10% of the Team X estimate, which for a complex multimillion-dollar space mission is extremely reliable. Team X handles a steady stream of customers from both within and outside NASA.
The JPL fieldwork and interviews I conducted focused on team members' interactions and use of technology in their work contexts. I observed the formulation of 13 different space mission designs, from requirements specification to completion. I found each mission design took from one to three sessions per week of about three hours per session. As part of this research, I interacted and conducted in-depth interviews with all 16 Team X members, as well as the team leader, a long-time customer, and the developer of the software tool enabling the team to specify mission parameters.
Networks in the War Room
How do only 16 people manage to produce a design proposal for an entire multimillion-dollar space mission, from launch, to orbit, to landing so quickly? First, they contact one another to discuss specific aspects of the mission. At the same time, another kind of networking, invisible to the casual observer, moves information around electronically through a custom-designed publish/subscribe system of spreadsheets. The combined human and electronic networks provide greater access to information than face-to-face interaction alone could provide.
For Team X members, years of working together have already established the social groundwork needed to communicate about mission design [6]. As a result, introductions are seldom necessary to preface communication or establish a context; the context is apparent from the activity in the room itself and the information on the linked spreadsheets.
When preparing any mission's design proposal, different parts of the human network might be connected. As soon as expertise is needed to solve a problem, team members seek out the source of the problem, possibly approaching colleagues already working on its solution, a customer with a question, or even a public display indicating an error. Others may join in the solution effort or stay at their desks calling out quick answers. Team X members routinely move between individual subsystem work, group work, and the orchestrated combined work of the entire team.
Team members constantly monitor information in the room both visually and aurally. Similar monitoring behavior has been observed in a variety of control room environments, including air traffic control and subway line switching rooms, where participants interact closely [1, 4, 7]. However, the Team X environment differs from these places in that it involves the monitoring of far more people and sources of information, including local group conversations, public conversations, public displays (possibly involving orbit visualization and spacecraft design), the linked spreadsheets, personal spreadsheets, the team leader, the customers, even the computer screens of neighboring engineers. One engineer reported he always keeps one part of his awareness tuned to what is being said anywhere in the room. This ability, performed while doing his own work or even being part of another conversation in parallel, seems to characterize all Team X members.
The cocktail party phenomenon, or when one attends to specific words in a noisy environment [2], is often observed in JPL, when keywords spoken aloud are relevant to an engineer's particular subsystem [2]. Hearing someone cite a subsystem by name, like power or propulsion, also gets other members' attention, suggesting the extreme seriousness and focus with which Team X members approach their roles.
Interdependencies
How do they select which information to monitor in such a noisy environment? Several reported developing "maps" of their own interdependencies, or roles, in the network. These maps refer to internal models of relationships on the team, including the other team members most likely to have the information they might need, as well as which team members are most likely to want the information others produce. The engineers know approximately how individual pieces of information build upon other pieces of information. For example, the telecom systems engineer needs information from attitude control systems, control data system, and ground systems for calculations to determine the type of telecommunication system that belongs on a particular spacecraft. Any change in a control data system calculation would affect the result of the telecom calculation. These interdependencies involve both the computer and the human-mediated information flow.
Adding to the intellectual challenge of learning these interdependencies is the dynamic nature of the war room network of people and electronic communication. Internalizing maps of interdependencies helps team members focus their personal monitoring on their fellow members whose interaction is most likely to affect their responsibilities in the overall design effort. The maps, combined with awareness of the work of other engineers, help reduce information overload by limiting the networking options in the design environment.
Electronic network. The team uses a publish-subscribe system of networked spreadsheets to exchange results produced by individual engineers, mostly involving components and calculations. The technology helps create a flexible electronic network in which team members publish information when ready and subscribe to information (assuming it's available) when needed. For a space mission proposal, the number of input parameters ranges from 11 to 519 per subsystem, with a median of 76 per person. These results are just the tip of the iceberg in mission design, as there can be thousands of underlying calculations and from 10 to 20 local conversations behind each published calculation.
The electronic network provides raw data, as supplied by engineers' calculations. The engineers process it through their human networking, that is, their face-to-face discussions. But because the electronic and human networks are highly interdependent, an engineer monitoring the conversation among other engineers and hearing a result is available might be prompted to subscribe to this particular result. However, results an engineer subscribes to through the spreadsheet system might also prompt further investigation by way of sidebar conversations to discuss the data and its context in the overall design.
Such information monitoring has helped the team recover from a number of software errors; for example, links to the shared spreadsheet can be broken if new items are typed in and, for some reason, the new items are then not published. On at least one occasion, as the team leader was discussing a new item and failed to see the results published, he immediately informed the engineer who then took action to establish the proper link. A new item not publicly discussed might also result in a broken link going unnoticed.
The cocktail party phenomenon, or when one attends to specific words in a noisy environment, is often observed in the war room, when keywords spoken aloud are clearly relevant to an engineer's particular subsystem.
Team leader. The team leader facilitates the social and electronic networks by monitoring all subsystems, telling team members to publish when they know others need their information and subscribe to information they know has been published and they need. These responsibilities have motivated him to internalize a complete map of the interdependencies of who needs what information when. He then might alter the public displays to show the specific data needed by an engineer to support the conversations he hears, such as an image of a spacecraft component when the subject is spacecraft design.
The team leader also keeps everyone on schedule and focused on moving toward the next design phase. For example, in one meeting I observed, he was able through a series of timely decisions to lower the cost estimate for a number of space probes from $20 million to $3 million. Although information spreads around the war room through word of mouth, the leader transforms separate discussions of errors into a group discussion. For example, his informed judgment determines when a problem becomes a crisis, such as when a spacecraft with solar panels would be traveling in an orbit that takes it through the shadow of a solar eclipse. He might tell the team to change some parameter of a mission design midway through a session, but it can take several announcements before the entire team responds to a major change by actually altering the mission specifications. This delay in getting the entire team to adjust its thinking, focus, and effort supports the idea that the team's engineers selectively focus on information, responding relatively slowly to comments not specifically directed at them.
Physical space. Each space-mission subsystem is mapped to a specific location in the room, that is, to individual workstations. Groups form as engineers walk over to other engineers to discuss technical issues, by remaining at their desks and talking to neighbors, or even by speaking across the room. At any point in time these engineers might be at their workstations (performing calculations or waiting for results); in sidebar conversations (solving problems); at the public display (addressing the information shown there); or at the customer's table (likely discussing mission requirements). The position of each team member in the room informs the other team members as to what might be happening with respective subsystems in terms of the overall design.
Activity among team members is always related to physical location [5], and each one's activity is visible to everyone else in the room. Thus, the physical arrangement of the entire group provides indication to everyone else as to the state of the human network, which in turn conveys information about a particular mission proposal's overall design status.
Seeing these people huddled around, say, a mission orbit on the public display, communicates that the orbit is the current focus of concern. Overhearing keywords in the conversation gives other engineers a fairly good idea of the problem. The physical co-location of the team members makes their human networking visible. The electronic networking is less visible, though it can be inferred by knowing the interdependencies and determining what information one has already received from the linked spreadsheets.
Limitations. The Team X environment is not for every engineer nor for every design team. Certain types of personality, such as being flexible and adaptable, are needed to be able to work in such a public environment. Most work is visible to all (except intermediate calculations), especially mistakes. Team members unable to adapt to an unstructured environment do not last long on the team. Moreover, they have to be able to withstand the stress generated by noise and time pressure; despite these distractions, one must be able to continually monitor the conversations, exchanges, design status, and changing mission parameters, as well as the team leader's remarks.
Almost all current Team X members report being mentally tired at the end of a session. Yet results from a questionnaire I administered to each team member showed the mean satisfaction level working in this environment to be quite high: 9.4 (sd=0.9) on a scale of 1 to 10 (high). One engineer described the experience as exhausting but thrilling, like riding a roller coaster. The key is to enjoy problem-solving in a highly social atmosphere.
A war room environment is most effective for teams responsible for tasks that are highly interdependent and when the relationships between individual participants are highly dynamic. When work is less interdependent and ad hoc, there is less need to participate in others' intermediate results, and the presence of other people working with, talking to, and watching each other might be distracting.
Delicate Balance
Extreme collaboration involves a delicate balance between electronic and social networks, though how to optimize the flow of information between them is an open question. Further automating the flow of information might relieve the personal stress reported by many Team X members. It might also reduce the potential for Team X human networking in which questioning assumptions, challenging values, and discussing options represent crucial aspects of designing space mission proposals. On the other hand, greater automation may free up time for yet more human networking.
Future research in collaborative war room settings needs to address the amount of social and task information people are actually able to use when designing a technically complex deep space mission. This knowledge would, however, prove valuable for designing new technologies that better support the networking that is an integral part of NASA mission design.
References
1. Bentley, R., Hughes, J., Randall, D., Rodden, T., Sawyer, P., Shapiro, D., and Sommerville, I. Ethnographically informed systems design for air traffic control. In Proceedings of CSCW'92 (Toronto, Oct. 31Nov. 4). ACM Press, New York, 123129.
2. Cherry, E. Some experiments on the recognition of speech, with one and with two ears. J. Acoust. Soc. Amer. 25 (1953), 975979.
3. Covi, L., Olson, J., and Rocco, E. A room of your own: What do we learn about support of teamwork from assessing teams in dedicated project rooms? In Cooperative Buildings, N. Streitz, S. Konomi, and H. Burkhardt, Eds. Springer-Verlag, Amsterdam, The Netherlands, 1998, 5365.
4. Heath, C. and Luff, P. Collaboration and control: Crisis management and multimedia technology in London underground line control rooms. J. Comput. Supp. Coop. Work 1, 1 (1992), 2448.
5. Kendon, A. Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge University Press, Cambridge, U.K., 1990.
6. Nardi, B. and Whittaker, S. The place of face-to-face communication in distributed work. In Distributed Work, P. Hinds and S. Kiesler, Eds. MIT Press, Cambridge, MA, 2001.
7. Suchman, L. Constituting shared workspaces. In Cognition and Communication at Work, Y. Engeström and D. Middleton, Eds. Cambridge University Press, New York, 1996.
8. Teasley, S., Covi, L., Krishnan, M., and Olson, J. How does radical collocation help a team succeed? In Proceedings of CSCW'00 (Philadelphia, Dec. 26). ACM Press, New York, 2000, 339346.
9. Williams, L. and Kessler, R. All I really need to know about pair programming I learned in kindergarten. Commun. ACM 43, 5 (May 2000), 108114.
Figures
Figure. The JPL Team X war room environment. Engineering subsystems include mission cost, thermal, power, structures, configuration graphics, systems engineering, telecom hardware and software, propulsion, ground command, mission design, attitude control systems, control data systems, instruments, science, and program management.
©2002 ACM 0002-0782/02/0600 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2002 ACM, Inc.
No entries found