Communications of the ACM

Communications of the ACM

Puzzles in Software Development Contracting

By Yossi Lichtenstein
Communications of the ACM, February 2004, Vol. 47 No. 2, Pages 61-65
10.1145/966389.966391
Comments

Organizations often choose to outsource software development in order to gain the expertise and scale unavailable internally [7]. This article presents an analysis of the economic aspects of external software development. Assuming that contracts should reflect risk and customer-specific investment, this research uses two economic theories to predict pricing, monitoring, and maintenance provisions in the analysis of 17 contracts in three corporations.

Software development contracts cover many technical and commercial aspects and are usually long and complex documents. To facilitate their analysis, this research focused on four contractual provisions [2] shown in Table 1. Two of the provisions, pricing and milestones, are related to short-term outcomes. The remaining two, maintenance provisions and effectiveness-based payments, reflect long-term outcomes.

Contract pricing reflects the short-term cost of a system, namely its initial development cost. Two basic techniques are used in practice: fixed price, and time and material. With fixed price, a price is predetermined for a specified system and services. The time and material technique requires that customers pay the vendor's costs plus a profit. Sometimes the two techniques are combined; for example, a fixed price may be set for an initial system, and time and material for its enhancements. This research examines the conditions for choosing between these pricing methods.

The second short-term outcome is the timeliness of the development; it is reflected in a contract by the project schedule and deliverables. These milestones should be frequent enough to prevent problems from remaining undetected; however, too-frequent milestones entail costly but unnecessary checkpoints [10]. This research examines the conditions that influence the intensity of milestones.

The long-term outcomes are not as explicit. Maintainability, or ease of system maintenance, is not reflected directly in contracts [2], but maintenance is. Typically, contracts allow customers to buy maintenance services that include corrections and adaptations to new data types and processing environments. Such services are usually priced as an annual fee of between 5% and 20% of the initial development cost. Contracts may also include a vendor commitment to develop enhancements on a time and material basis with predefined hourly rates. This research examines how customers decide between buying external maintenance services and maintaining the system internally.

The ultimate outcome of software development—long-term effectiveness—is usually not reflected directly in contracts [2]. However, some contracts tie the final payment to an effectiveness measure. As with the previous outcomes, this research examines the conditions for effectiveness-based payments.

The study focuses on the economic criteria that influences decision making regarding these four contractual provisions. Other dimensions, such as organizational aspects, help shape the relationship between customer and vendor, but they are not directly addressed in the current research. We use two theories of contracting: agency theory [4] and transaction cost economics [11]. These theories have been strongly supported by empirical research [4, 9]. The next paragraphs contain brief descriptions of the theories, their application to software development, and the predictions they suggest. This information is summarized in Table 2.

Agency theory. This theory assumes an uncertain venture owned by a principal and performed by an agent whose actions are not fully observable. Outsourcing of software development fits this description well, as uncertainty is significant and performance is difficult to measure. A linear agency model [5] demonstrates the trade-off between incentives and risk sharing; if the venture is sold to the agent, he exerts maximum effort and bears the full risk, while if the agent is paid a flat fee, he bears no risk and has no incentive to exert effort. The model demonstrates that a combination of a flat fee and performance-tied payments is optimal from the principal's perspective. The model also shows that higher risk reduces the likelihood and size of performance-based payments, and that monitoring the agent's actions and tying payments to the monitor's signal becomes more valuable when risk is higher.

We interpret fixed-price contracts as performance-based with respect to the initial development cost, since the vendor benefits from cost savings and pays cost overruns. Namely, fixed-price contracts reward the vendor for his cost performance and are thus performance-based. Consequently, the main proposition of the agency model is that fixed-price contracts are likelier for low-risk projects.

The model is also useful for studying milestones, which can be interpreted as monitoring devices by which the customer observes the vendor's effort. In particular, payments tied to milestones can be seen simply as rewards linked to the monitor's signal. Consequently, the model predicts that milestone-tied payments will be more intense for higher-risk projects.

Lastly, when considering long-term effectiveness, payments dependent on the effectiveness of the system can be seen as performance-based incentives. Here, the model's main proposition predicts that effectiveness-based fees are likely for low-risk projects.

Transaction cost economics. This theory explains that firms decide either to buy or make a good by balancing production and transaction costs. The latter include the costs of negotiating, consummating, and safeguarding a contract. The theory predicts that firms are likely to make a good when its production entails specific investment, and when the good must be produced at frequent intervals [11]. The specific investment reduces market economies of scale, reduces the number of potential suppliers, and may allow a vendor to lock in customers and charge them excessively. Thus, the theory predicts that when production is specific and frequent enough to justify investment, the customer is more likely to produce the good internally.

We interpret fixed-price contracts as a "buy" decision, and time and material contracts as relatively close to a "make" decision. The reason is that a fixed-price contract buys a system, while time and material only buys resources to make the system. Thus, transaction-cost reasoning predicts that fixed price is likely for projects that do not entail high customer-specific investment. If such investment is needed, the customer will make the system by contracting on a time and material basis.

Transaction cost economics can also be used to analyze the maintenance decision. A customer decision to maintain an in-house system is a "make" decision, while buying maintenance services externally is a "buy" decision. This simple interpretation allows for the prediction that infrequent and non-specific maintenance will be contracted to the vendor, while frequent and specific maintenance will be performed internally.

Back to Top

Methods and Data

The research focused on contracts as the evidence regarding the practices under study. Three of the largest firms in Israel granted full access to their software development contracts. The firms were in the services, finance, or production industries, and information systems were considered as crucial support, strategic, or support activities, respectively. The companies had between 1.5 and 3.5 million customers and between 10,000 and 30,000 employees. The information systems departments of the three companies were between 200 and 850 strong, with annual development budgets of $35 to $45 million, and with 40% to 80% of this budget going toward outsourcing. The three organizations had similar contracting procedures, with two of the firms using the same contracting guidebook.

During the study period, the three companies had 17 ongoing contracts. The projects included the development of managerial information systems as well as transaction processing systems. The most expensive project was the development of a logistic management system for approximately $580,000 (not including an additional $245,000 spent on packaged software), with eight programmers working for 21 months. The least expensive project was a $38,000 directors' management system, with three programmers working for six months. The median project was a payroll decision support system for $130,000 (plus $245,000 for packaged software), with eight programmers working for four months.

For the purposes of contract analysis, I recorded the pricing technique, whether it was fixed price, time and material, or a combination thereof. To measure milestone intensity, I recorded the number of milestones tied to payments, the size of the tied payments and damages, and the level of detail of the specification for milestone deliverables. Maintenance provisions were recorded, as well as customer intentions to either maintain the system internally or via the development vendor. Finally, for effectiveness-based incentives, the time and size of payments after system deployment were noted.

In addition to the contract analysis, I conducted structured interviews with the contract managers, mainly to assess risk and specific investment. To measure risk, I adapted a questionnaire that considers five components of development uncertainty (novelty, scope, expertise, complexity, and organizational environment) and the magnitude of potential loss if the project fails [3]. I added questions about outsourcing uncertainty, in particular the vendor's size and reputation and the duration of customer-vendor relationship. Risk was then calculated by multiplying uncertainty by potential loss. To prevent hindsight bias in assessing risk, the study was limited to ongoing projects in which the systems were not yet deployed.

Several measures were used to assess specificity, or the degree of customer-specific investment. The objective measures included the number of links to other systems, the price ratio between custom-made and packaged software, and the numbers of potential suppliers and customers. Subjective measures of specificity included an assessment of the learning needed to define the system, and the delay expected if a new system analyst would be hired. Finally, the project managers assessed maintenance frequency in terms of tasks and cost.

Back to Top

Puzzling Findings

Although considerable variety existed in terms of the risk and specificity of the projects, all 17 contracts were fixed price. None was priced on a time and material basis. In contrast, the variability in milestone intensity was considerable, between one and eight milestone-tied payments per contract, with milestone-tied damages in about half of the contracts. However, the prediction that milestone intensity would be correlated with risk and specificity was not supported.

All contracts included vendor obligation to maintain the system for a fee, but only 11 of the systems would be maintained by the vendor, while six would be maintained internally. The prediction that low specificity and maintenance frequency would increase the likelihood of external maintenance was not supported. As for the fourth outcome, effectiveness, only five contracts included payments after operational deployment, with none using systematic measurement of effectiveness. The prediction that low-risk projects would include effectiveness-based payments was not supported.

Since the predicted structures were not discovered, I looked for alternative patterns. The number of milestones was positively correlated with the development time. About 40% of the milestone variance was explained by the project duration—apparently, milestones were used to divide the development period into stages of two or three months each. This is consistent with agency theory; however, time—a simple parameter—was used to set milestones, while the more complex concept of risk was ignored. Also, decisions regarding maintenance were related to the magnitude of potential loss. Internally maintained systems had significantly larger loss scores, in particular regarding potential loss related to daily operations and market share. This is consistent with transaction cost arguments, as the vendor may lock in the customer to a critical system and charge highly for its maintenance.

The failure of this research to establish the predicted patterns in the study sample may relate to study weaknesses, such as the reduction of software development to four narrow contractual provisions. I recorded the number of milestone-tied payments, for example, but did not include mechanisms such as informal meetings or user participation in the study. However, the aspects included in the analysis are central to outsourcing and to the relations between customer and vendor. Another study weakness may involve the small number of contracts studied, which allowed minimal statistical testing. However, several statistically significant patterns were found that point to the credibility of the study data. A final factor limiting the generality of these results is the Israeli context. Israeli managerial practices are similar to those in the U.S. and Europe, but these results may reflect reliance on informal relations with vendors in the relatively small Israeli software industry [1].

An alternative explanation exists for the inability of this research to support the predictions: current outsourcing practices may fall short of the sophistication prescribed by the theories. Given the wide acceptance of the agency and transaction cost theories [4, 9], the significance of the research is in applying theory-based critique to the contracting practices observed, as summarized in Table 3 and discussed here.

Back to Top

Applying Theory in Practice

The three firms appeared to use a limited range of development arrangements: they either develop internally or outsource on a fixed-price basis. The failure of these research sites to utilize time and material contracts is potentially inefficient, as high premiums may be required for risky fixed-price projects, and internal development is undertaken even if the firm lacks the required expertise.

The failure to utilize time and material contracts is further puzzling because 11 of the systems were to be maintained by their respective vendors, meaning that system enhancements would be separately contracted on a time and material basis using the hourly rates set in the development contracts. Namely, the same firms that refused to contract initial development on a time and material basis accepted it for what is typically the major element of cost in a system's life cycle [6].

These study findings suggest that firms should use the full range of pricing techniques, such as time and material and cost sharing. As customers already buy enhancements on a time and material basis, they should use these smaller ventures to build trust in the vendor time and material accounting [8], and to allow such pricing arrangements for full projects.

The second prescription concerns the intensity of milestones and risk. While the study did not support the prediction that milestone intensity was related to project risk and specificity, it did correlate with project duration. Although setting a milestone every two or three months seems reasonable, it is not the most efficient technique. For example, the investment in monitoring a one-year project with low risk is similar to that of monitoring a one-year risky project. Clearly, it is preferable to invest more in monitoring the risky project.

There was no reason to doubt a priori the firms' abilities to react to risk, since all three firms used standard project management techniques to control risk, and the project managers were well aware of software development risks. But in retrospect, the negative study result raises a question as to whether appropriate reaction to risk can be achieved without an explicit risk management methodology, which none of the three firms employed. The results of this study suggest that firms should apply a formal risk measurement and management procedure, and that they should use it in setting the control and monitoring intensity of both external and internal projects.

The third prescription concerns internal and external maintenance. Although maintenance frequency and specificity were not found to predict how systems would be maintained, the finding that critical systems would be maintained internally is consistent with transaction cost economics. The suggestion here is for firms simply to take maintenance frequency and specificity into consideration. In addition, contracting enhancements on a time and material basis may not be optimal; instead firms may want to consider cost sharing.

Finally, only a small degree of effectiveness-based pricing was noted among the firms studied. It is true that effectiveness can be difficult to measure, since its achievement depends on both customer and vendor. However, the agency literature teaches that some performance-based payments are essential to induce effort, and firms would be wise to experiment with effectiveness measurement and effectiveness-tied payments.

Back to Top

Conclusion

I would like to reframe my results and prescriptions in the context of risk and incentives. A contract addresses both short- and long-term risks. Vendors undertake short-term risk through fixed price and milestones-tied payments. But customers undertake the full long-term risk, since most maintenance is undertaken on a time and material basis, and payments are not tied to long-term effectiveness.

This asymmetric risk sharing is inefficient. Theoretically, risks should be shared more equitably in order to provide incentives to reduce them. The customer should induce the vendor to reduce long-term risk and to achieve long-term effectiveness. But such incentives were not observed in this study; in fact, efforts made by vendors to reduce risk related to cost and intermediate deliverables may reduce their effort to achieve maintainability and effectiveness.

Long-term incentives may still exist, since vendors wish to keep their reputation and gain future contracts—but incentives within each project may be a more efficient way of providing a comprehensive set of incentives. In particular, the trading of some short-term risk for long-term risk—for example, trading cost sharing for effectiveness-based payments—may be a simpler and more direct way to induce effort in external software development.

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2004 ACM, Inc.

---

No entries found