Communications of the ACM
Home/Magazine Archive/December 2007 (Vol. 50, No. 12)/Moving Beyond User Participation to Achieve Successful.../Full Text
Communications of the ACM
Moving Beyond User Participation to Achieve Successful IS Design
In a ham and egg breakfast, the chicken is involved, the pig is committed. This old saying may hold the key to understanding the high failure rates of large-scale software development and implementation projects. Indeed, there is a fundamental difference between participation and engagement. Our research suggests that work force commitment and true participation (that is, engagement in the process) are often difficult to achieve in large-scale software design and implementation projects. For this reason, we find it becomes imperative to focus on the timing of user participation, not simply to advocate and plan for their involvement. We suggest that rather than fighting human nature by trying to force the participation of user groups throughout a project, we should broaden our thinking about development and implementation methodologies to reflect what happens in practice. We find that in practice user participation can be most powerful after 'go live' when users are truly engaged.
Involving users in software project initiatives has been frequently listed as a critical factor in the successful implementation of software.1 Following the long tradition of research in user involvement and participatory design, the belief that users should take part in the design of the software applications with which they will be working is recognized as being good practice. This is reflected in the wide spectrum of methodologies in use today (for example, prototyping, agile programming). The principles of participatory design stem from the belief that involving users provides multiple benefits. These include increased user accountability for the system's design, thus resulting in work force commitment, reduced employee resistance to change, and increased job satisfaction. Additionally, a participatory design approach is expected to help mediate power relations among different stakeholders and facilitate organizational learning by producing valuable information about the organizational change process.
The motivations for adopting a participatory design approach are compelling, and its adoption is clearly beneficial. Consider the recent example of Nestlé's ERP implementation:
"... The [ERP] rollout had collapsed into chaos. Not only did workers not understand how to use the new system, they didn't even understand the new processes... [The project manager] says her help desk calls reached 300 a day. 'We were really naive in the respect that these changes had to be managed,' she admits now.'"
This example highlights the potentially destructive effects of neglecting users and their concerns. The effective management of user participation has long been advocated as a solution. In spite of this, user resistance to new systems remains high, and their recalcitrance can derail multimillion dollar development efforts, placing firms in a vulnerable and dangerous position. While sometimes user participation is nothing more than a window dressing effort aimed at gaining buy-in, most organizations honestly attempt to engage users by adopting participatory design, prototyping, and similar approaches. For example, during our own study of a multimillion dollar ERP project, 'super-users' were assigned full-time to the project and their positions were back-filled for the initiative's duration. In addition, those members of the user community who were not directly involved in the project were invited to attend Q&A sessions throughout the project where feedback from the audience was sought. In this case the precepts of participatory design were genuinely followed.
These efforts notwithstanding, many CIOs and project teams remain puzzled by the still high rates of failure, and are at a loss as to why attempts to involve users fall short of the goal of successful implementation and use. Late delivery of software, over-budget implementations, scope creep, user backlash, limited technical functionality, and incomplete business process redesign remain an all too common occurrence.
As different as these manifestations of failure are, they all stem to some degree from users' resistance to, or outright rejection of, the software being introduced. User rejection or requirements changes slow down implementation, add costs, and lead to a lack of system usespelling project failure. Participation, conceptualized as the involvement of users in discussions over time to elicit feedback and commitment, promises to offer some solutions. Similarly, prototyping, conceptualized as the development of a working model in order to obtain user feedback about the system and interface design, also is potentially helpful. But, as we argue here, the success of these approaches is predicated on users becoming truly engaged in the process. This is far from an automatic occurrence, even when the methodologies are adopted and followed.
Here, we offer an explanation of the difficulties associated with current participatory approaches and present some guidance on how to overcome them. Our insight emerges mostly from a longitudinal study of an enterprise systems implementation at an Ivy League university we call ILU, involving 129 semi-structured interviews with 53 stakeholders. This data is combined with the analysis of a number of other project failures from public sources and a reading of the substantial body of research in this area.
Moving Beyond Participatory Design: It's About Time
Our research indicates that because users are busy at work and their attention is captured by immediate responsibilities, they will generally not become fully engaged in analyzing and evaluating new systems, even when the precept of early user participation is followed. Rather, as it is human nature to become committed only when one's sphere of work is impacted, busy users tend not to fully engage until the system's impact on their working life is apparentgenerally when the system 'goes live.' Prior to this phase, attempts to increase user participation are helpful, but only partially effective. Further, in an era of information overload, individuals are under constant stimulation and must filter messages in order to avoid being overwhelmed. Thus, the motivation to process a given message is determined by its relevance to the receiver at the time of receipt. This phenomenon is well documented in the psychology and marketing literature, and is generally referred to as elaboration likelihood, which suggests that individuals must be both motivated and able to process information in order for it to become salient and spur them to action. For example, on a daily basis most people pay little attention to car advertisements. Conversely, the few who at any time are in the market for a new car pay close attention to the ads and even examine cars they see on the road. This is because for the latter group cars are a topic of interest, thus car advertisements and cars on the road represent stimuli that warrant attention.
Software development projects become salient to users when the output affects their daily lives and requires them to change their work practices.
A similar dynamic occurs with respect to software design and implementation projects. For end users caught in their day-to-day activities, involvement in new software design is often treated as no more than a distraction. In a software development project a user may be motivated by how the new technology impacts his or her working life. In other words, software development projects become salient to users when the output affects their daily lives and requires them to change their work practices. Before that time, users are likely to have little incentive to be fully engaged. This is not due to an unwillingness to be involved, but it takes significant cognitive effort to envision what the end product will be and, more importantly, how it will change work practices and affect the users' own domain. As a consequence, the project is often low in salience and does not capture users' full attention. To the design team it appears the project is being well received, while in reality users are merely acting as if they are engaged in the design effort. But as we show in this article, there is a temporal element to elaboration likelihood where engagement is determined by how closely the information we are given will impact our daily lives. Thus, when project completion is imminent and the reality of new work practices becomes apparent, users begin to closely evaluate the new system and significant issues are raised, often leading to the failures discussed earlier.
We experienced this firsthand when serving on the faculty advisory board on information technology at our own institution. Cornell (not ILU) recently attempted to migrate to a new course numbering scheme necessary to support the installation of an ERP application. As the introduction of the new scheme became imminent, some faculty on the advisory board raised a number of issues regarding incompatibility of the new numbering scheme with many of the uses of these numbers. Interestingly, six months earlier this same board had been briefed by the project manager about the system development and its progress, including the new numbering scheme. At that time, the board seemed to see only minor problems with the changeover, acting as if they saw no problems to project management. Thus, from the perspective of the project manager, the faculty looked as though they were on board from the first briefing because they were silent about potential problems. We contend this was not the case. Rather, faculty did not complain when initially briefed on the conversion of course numbers because the message was not salient to them at that time. They did not see the potential problems and changes to their work practices because they were focused on other more pressing issues, and the new system roll-out was far into the future and still in flux. Only later did the conversion become salient when the changeover to the ERP system was imminent and they had clarity about the new work practicesit was then the faculty raised major concerns.
Insight from our research suggests it is not enough to involve users. Software design teams must give careful consideration to the timing and focus of user participation. Specifically, what seems crucial is the point in time when users are involved (recognizing that earlier is not necessarily better) and their ability to conceptualize how the system will impact their work practice becomes clear, thus allowing them to focus on the critical issues the system engenders. Equally important is engaging users who are not directly involved in the system as members of the project development team. We find that it is these users who can make or break the success of new system implementations.
To realize the benefits of timely user involvement it is imperative that users be considered experts, and analysts develop the skills necessary to act as translators between those who do the work and those who design the software.
Lessons from the Field
At the inception of the ILU ERP project, the user community was promised improved work practices as a result of what was termed the "Systems' Modernization Initiative." The project team strictly followed the tenets of participatory design, adding representatives of the user community to the team. The project team also organized several targeted user-group presentations during the two years of the project initiative, but turnout at these meetings was minimal. Some of the users who did attend the targeted presentations were resentful of the meetings, seeing them as a waste of time, and too abstract to provide any meaningful feedback to the project team. The lack of concerns raised by the users at these information sessions was interpreted by the project team as an indication of their engagement with the initiative. This is revealed by the project manager, who interpreted a lack of dialogue as a sign of approval, saying "We're doing great, these guys are on board."
Once the ERP went live, end users in general were surprised and unhappy with how the system forced them into new routines they considered inadequate. It was only then that they expressed their dissatisfaction with the proposed ERP and worked to force a costly shift in design. A senior manager at one of ILU's professional schools captured this point: "So they spent millions [of dollars] and didn't meet the business needs of the community; that's hard to take and I think they are having a hard time admitting they were wrong... That's why we are building our own." This repositioning led to post-implementation challenges that upset interpersonal relationships, increased the project budget, and delayed the acceptance of the ERP into the work environment.
Project leaders were resentful of users and accused them of an 11th-hour interest in the project, rather than a dedicated commitment to the software design. The project leader aptly summarized this resentment: "We asked all along for feedback from them but no one said anything, and only now are we finding out the problems." In turn, users were angry that the promise of "improved work practices for all members of the community" was not met and that the project team was now impatient when dealing with their post-installation concerns. Despite dedicated super-user involvement in the project and communication with the ILU community, modifications were being made to the software, its outputs (such as reports), and accompanying business processes three years after going live.
Avoiding the no-win situation: What should the project team do? So what is one to do when every effort is made to create a system that will be accepted and used in the organization, only to find their efforts are underappreciated and contentious? We offer the following guidelines.
Think differently about how to involve users. It is difficult to garner user attention throughout a complex and lengthy development project because end users are busy. Thus, the project team must think creatively about how to foster users' active participation. When seeking participation, one is more likely to engage users on topics that have salience for them at that time. Therefore, we suggest a strategy of engagement that takes as its starting point the present moment by asking users about their day-to-day work activities. The present moment is salient to the users and it is therefore simpler for them to adopt it as their departure point. It follows then that the design team should elicit users' stories about current work practices and pay particular attention to what they have identified as either challenging or successful ways of working. This approach is also valuable because it shifts the focus from what the software will do toward how individual users will work with the software.
Broaden the skill set of system analysts. The guideline focuses on asking users to describe how they carry out their activities. This leads to a narrative of current work practices. Interpreting these narratives is a skill not generally taught to system analysts, but one that is increasingly recognized as crucial. Moreover, research has shown that software analysts often perceive themselves as experts. As a consequence, they tend to unconsciously devalue end-user storytelling and fail to internalize the stories being offered. To realize the benefits of timely user involvement it is imperative that users be considered experts, and analysts develop the skills necessary to act as translators between those who do the work and those who design the software.
Enact a modified system development life cycle. At ILU, despite efforts to include users on the project and create interest among the wider community, the development team was only partially successful in creating a working information system that was accepted and used by ILU employees. Upon distribution of the system to the user community recalcitrance was high, taking the project team by surprise. This resulted in a post-implementation redesign of system functionality and business processes in an attempt to stave off full-scale rejection of the software. While the design team followed the basic tenets of participatory design, they focused on who to involve while neglecting to evaluate when such involvement should occur.
To this end we acknowledge it will be difficult to fully engage users before the software begins to affect their daily lives, which are generally at 'go live.' Yet, we are not suggesting there is no value in user involvement via prototyping or phased roll-out techniques. When properly implemented, these techniques can help increase communication, provide some valuable feedback in the design process, and generally improve goodwill on the user side. But it is critical to recognize that trying to force engagement of user groups throughout a project goes against human nature. We should therefore expand our thinking about the stages of the systems development life cycle, and incorporate this broadened perspective into whichever methodology is used. We need to recognize that implementation extends beyond "flipping the switch." Legitimizing the post-implementation activities that are often kept as a shameful secreta sign of project failurewill help to manage expectations and avoid much of the conflict that erodes trust between the user community, project champions, and the development team by more naturally mimicking human behavior. This realization is one of the key insights of this article and an accurate description of reality, rather than yet another idealized model of theoretically sound, but difficult to apply, ideals against which you are told to benchmark performance.
Conclusion
Anyone who has been involved with systems development realizes the dedication and intellectual energy required to create a working information system. This article recognizes such efforts and proposes changes to user participation activities that align human behavior with efforts to gain user "buy-in" and commitment to a new system. It is time to speak honestly about the gap between our intentions to build working systems and our ability to do so in practice. This gap is typically not caused by a lack of effort on behalf of developers or users, but rather is the result of misdirected efforts. The systems development and implementation process will continue to be overly challenging if we work against the tide by trying to make users fit our theories of how and when they should participate in development initiatives. Instead, we suggest catching waves with users at opportune moments, working to hear what they are saying, and then adjusting our, and their, expectations about when a system is completed.
References
1. Alvarez, R. and Urla, J. Tell me a good story: Using narrative analysis to examine information requirements' interviews during an ERP implementation. The Data Base for Advances in Information Systems 33, 1 (2002), 3852.
2. Denning, S. Telling tales. Harvard Business Review 82, 5 (2004), 122129.
3. Mumford, E. Redesigning Human Systems. Information Science Publishing, Hershey, PA, 2003.
4. Petty, R.E. and Cacioppo, J.T. Communication and Persuasion: Central and Peripheral Routes to Attitude Change. Springer-Verlag, NY, 1986.
5. Scott, S.V. and Wagner, E.L. Networks, negotiations, and new times: The implementation of Enterprise Resource Planning into an academic administration. Information and Organization 13, 4 (2003), 285313.
6. Worthen, B. Nestlé's ERP odyssey. CIO Magazine, (May 15, 2002).
Footnotes
1The Standish Group's annual CHAOS reports have ranked user involvement as the 1st (1994) and 2nd (2000) factor for successful IT project success; www.standishgroup.com/.
©2007 ACM 0001-0782/07/1200 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2007 ACM, Inc.
No entries found