Communications of the ACM

Communications of the ACM

Critical Risks in Outsourced IT Projects: The Intractable and the Unforeseen

By Hazel Taylor
Communications of the ACM, November 2006, Vol. 49 No. 11, Pages 74-79
10.1145/1167838.1167840
Comments

We continue to be assailed with reports about how difficult software projects are to manage within time and budget, and about the costs of failure or underdelivery on these projects [10]. Many firms have turned to outsourcing, either contracting specialist software firms for custom development work or choosing to install a packaged solution in an attempt to minimize or transfer the risk, but these options, too, seem to be fraught with difficulties [11]. At the same time, authors argue that the risks that threaten IT projects are well known, and prudent risk management should ensure successful outcomes [7]. Why then are the problems still continuing?

Much of the research to date has centered on identifying risks that can threaten in-house software development projects [7]. A recent survey of MIS directors has highlighted six factors (use of an inappropriate methodology, lack of customer involvement, lack of formal management practices, dissimilarity to previous projects, project complexity, and requirements volatility) that these experienced managers rated most important in risk assessments of their projects [10]. Most of these risks also apply to outsourced custom or package projects, but recent research suggests that there are some unique risks for clients of outsourced projects, arising from possible difficulties in controlling the vendor involvement [2], and in managing working relationships between vendor and client [5, 6]. As organizations increasingly utilize outsourced solutions for their systems needs, understanding sources of problems in these types of projects becomes increasingly important. The study discussed here is unique in that it focused on outsourced projects from the vendor's perspective and identified key risks that are difficult to manage and hence especially important for both vendors and clients to be aware of. Two risks in particular, overoptimistic schedules and budgets and inflated client expectations, are critically important for both vendors and clients considering outsourced projects. Both of these risks arise from the vendor's desire to win business in a highly competitive marketplace.

Knowing what risks may threaten a project is the starting point for ensuring those risks do not evolve into full-blown problems, since once potential threats to a project have been identified, mitigating actions can be taken to reduce the likelihood they actually occur. Yet clearly, since problems with IT projects persist, simply knowing a comprehensive list of risks is not enough to ensure success. It is more important to know which of the many possible risks are most likely to cause problems, either because they are difficult to mitigate or because they are difficult to anticipate. In particular, when conducting a pre-project risk assessment, two types of risks—intractable and unforeseen—are especially important to watch. Intractable risks are those risks that resist mitigating actions, and still impact the project despite the manager's best efforts to address them at the start. Unforeseen risks are typically overlooked or simply don't seem likely to happen at the risk assessment stage, so that no action is taken to mitigate them. Both intractable and unforeseen risks pose a significant threat to project success even on projects that are subjected to a rigorous pre-project risk assessment.

In this article, I focus on these two types of risk in outsourced projects. To learn more about which of the many potential problems identified at the start of projects typically prove to be difficult to manage, and also to uncover those problems most typically unanticipated at the start, I interviewed 25 experienced project managers working with outsourced projects in Hong Kong, using an in-depth critical incident interviewing technique [4, 9]. The respondents discussed specific risk situations related to 39 projects, including 10 "troubled" projects they had managed. They detailed both the risks they had identified at the start, and the problems—anticipated and unanticipated—that came up during their projects. (See "How the Study was Conducted.")

Back to Top

Intractable Problems

As shown in Figure 1, the most likely problem to arise, even after being anticipated and addressed with mitigating actions, was schedule and budget management, often compounded by the second and third most common problems of vendor staffing issues and difficulties arising from the newness of technology. All three of these problems were seen by respondents as having their underlying cause in inadequacies in the pre-sales analysis, resulting in either an overall shortfall in budget or time allowance, or in an underestimation of the skill set required or of the extra work involved in coming to grips with new technology. In all eight of the projects where schedule and budget problems still arose after mitigation, the managers believed they had been asked to take on an overly ambitious project schedule, committed to by the pre-sales team.

This highlights a very important issue, namely the tension between pre-sales and implementation stages of outsourced projects. Project managers viewed the desire of vendor sales teams to win the bid while still remaining within their own limits in terms of pre-sales work costs as an underlying cause of many subsequent problems in outsourced IT projects. As one respondent noted: "I can say that almost across the board for IT personnel we oversell, we promise more than we can really do. We try to get the business a lot of the time—of course we are vendors, we provide services, but even for in-house MIS organizations, we over-promise. IT people do not give a real picture to the users about the project challenges, and I think that's a major, major problem."

On the surface, it might not seem so surprising that implementation managers would try to shift the responsibility for problems in their projects away from themselves. But there was a strong consensus among respondents that the issue of overselling or over- promising was almost endemic in the industry, with the pressures to win business, and the desire to satisfy customers often outweighing more cautious responses to client requests for proposals. Indeed, respondents were honest about their own tendencies to present the "best picture" when they themselves had been involved in pre-sales work.

The best way to avoid underestimation of project budgets and schedules is to do the pre-sales requirements analysis as thoroughly as possible, but this is costly, particularly if the bid is not successful. Even if a thorough pre-sales analysis is completed, the final bid may need to be cut in order to make it competitive. Some readers might assume this problem would most likely occur with small vendor firms, since they are most likely to be hungry to win a bid. However, my data suggests that large firms are not immune to this problem. In fact, six of the eight projects where schedule or budget continued to be a problem after mitigation were large firms that had well-established procedures aimed at minimizing potential problems flowing on from the pre-sales stage.

This apparent failure of mitigation actions taken at the pre-sales stage to prevent the occurrence of schedule and budget problems highlights the critical importance of this risk. Respondents suggested the most effective approach to address this issue was to separate the requirements specification part of pre-sales work into a standalone, chargeable consulting activity. This pre-project partnering approach has been recommended by other researchers [3] as an effective strategy for improving subsequent project performance. It has the advantage for the vendors of supporting them in gaining an in-depth understanding of the client's requirements, while at the same time allowing the clients to assess the type of working relationship they could expect from the vendor without having to make a full commitment to the vendor for the entire project.

Back to Top

Unforeseen Problems

Given the tendency to "paint a rosy picture" to clients at the start of projects, it is not surprising that the most common unforeseen problems tended to be client relationship problems, as shown in Figure 2. Client relationship problems appeared unlikely to be expected at the start, partly because signs that such problems might occur are typically not evident early in the vendor-client relationship, and also because their intangible nature makes them difficult to quantify and assess. Respondents typically expressed some surprise that these problems had arisen during the course of a project and it seemed that such problems were difficult to identify at the early stages of the project, when both sides were eager to negotiate a deal and as such were, perhaps, on their best behavior. Indeed, one respondent commented: "It's almost like a marriage, it starts off very, very happy, everybody has a rosy picture about what's coming down the pike ..."

With the exception of client expectations, the set of problems that occur unexpectedly in projects is quite different from those that arise in spite of being anticipated and mitigated. The predominance of client relationship problems in Figure 2 supports the need for strong measures to meet the relationship needs of client staff [6]. Clearly, project managers would be well advised to pay attention to relationship issues both with their client and within their own team at the start of the project, and to take steps to ensure those relationships are well founded right from the start.

The client expectations factor was also a key factor in "troubled" projects, as seen by a number of respondents as one of the key areas for managing risk.

Back to Top

Troubled Projects

The 10 troubled projects were of particular interest, since they came close to failure, and the assessments of these projects by the highly experienced managers brought in to rescue them give significant clues about critical factors that can make the difference between success and failure. The table here shows the top six problems these troubled projects managers believe had sent these projects off the rails prior to their taking over.

The six key risks noted here were highlighted as critical factors causing major problems in troubled projects. Three of these risks featured highly among those typically anticipated at the start of projects, while the remaining three occurred frequently as unexpected problems in projects. Control of these six key risks was considered by respondents to be crucial for successful management of risk in their projects.

The appearance of client expectations in all three problem sets is particularly significant. Customer satisfaction is related not only to what has been delivered and how well the process went, but also to what was expected in the first place. Indeed, the degree of disconfirmation of expectations (the difference between outcomes and expectations) strongly correlates with the level of customer satisfaction, as noted by Szymanski and Henard [8] in their recent meta-analysis of customer satisfaction studies. So if expectations are high to start with, and the outcomes are not quite as high as those expectations, then customers will ultimately be dissatisfied, even if final functionality and performance is a good match with requirements. As Szymanski and Henard comment, "negative ramifications ... can result when firms overpromise and underdeliver." Customer satisfaction is obviously extremely important for vendor firms since it has a major impact on their reputation and can significantly influence future business, particularly since new clients are likely to ask for reference sites when assessing vendor bids [6].

While the factors identified by these troubled project managers are shown as separate factors, the process of identifying and coding each factor separately actually obscures the view of the project managers that, in fact, these factors were highly interrelated. Poor schedule and budget management was described as a key factor in causing problems in these projects, and in five of the 10 projects this was attributed to an original underestimation or underbidding on the part of the pre-sales team. The following comment by one project manager is typical: "They significantly underbid. The estimate was wrong [because] the complexity of requirements was not fully understood."

This respondent went on to explain how the situation was compounded by serious deficiencies in the estimation of the vendor staff skills required to complete the project: "It was grossly undersized ... The original plan called for just one guy with skills and knowledge of that package. We ended up by having about six or seven people, brought in from overseas."

The failure of the vendor staff to fully understand the client requirements at the pre-sales stage also contributed to both the underestimation of schedule/budget and of required skills: "We tried to put a package solution in where a new development—a roll-your-own solution—would have been more appropriate .... You couldn't take any standard package off the shelf and try to fit it to [these requirements]."

This led to difficulties in controlling change requests from the client, since, from the client's perspective these were not change requests but legitimate requirements of the project:

"The requirements were not so much changing but being reinterpreted at times. ... So you could say there may not have been additional requirements, but clarification of how the requirements should be dealt with."

The compounding interrelationship of problems described here was characteristic of troubled project problems. Most often, the requirements had not been fully understood at the pre-sales stage. As the complexity of the requirements became clearer, typically performance against schedule deteriorated, setting up a spiral of declining team morale, which caused further performance deficiencies. Often underlying all of this was the failure of the pre-sales team and the original manager to set and control client expectations at a reasonable level.

Back to Top

Conclusion

This research has highlighted certain critical risk factors that both vendors and clients should be aware of when entering into an outsourced agreement for an information system, and has suggested key strategies for addressing these risks. In particular, schedule and budget risks can become intractable problems, particularly if they arise from underbidding by vendors or by clients selecting a vendor partner on price alone. Both parties should consider pre-partnering arrangements to fully develop requirements specifications before entering into an implementation contract, as a strategy for minimizing these intractable schedule and budget problems. Unforeseen problems typically involve relationship issues that can crop up unexpectedly at any stage of the implementation, even in projects starting off harmoniously. The prominence of client expectations as a problem in both the intractable, unforeseen, and troubled project categories suggests that educating the client to have a realistic expectation of how the project will progress is a key strategy for vendor project managers in ensuring client satisfaction at the close of the project.

Clients, on the other hand, should take care to ensure their own expectations are reasonable, and again, the pre-partnering approach enables both sides to develop a clear understanding both of the complexities of the project requirements and of the likely performance of the other partner. Both clients and vendors should use this opportunity to evaluate the other party's likely working style before making any major commitment.

Back to Top

Back to Top

Back to Top

Back to Top

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.

---

No entries found