Communications of the ACM
Home/Magazine Archive/August 2006 (Vol. 49, No. 8)/Defining and Contributing to Software Development.../Full Text
Communications of the ACM
Defining and Contributing to Software Development Success
The use of the traditional, and somewhat simplistic, definition of software project success (completing a project that meets customer/user requirements on time and within budget) has been an important factor affecting widely cited statistics of failed projects (the Standish Group's studies from the mid-1990s come to mind), which can then be "...used as fear-based advertisements for consultant services or quick-fix techniques/tools" [7]. Further, Linberg suggested this traditional definition of project success "may be too narrowly defined and may create negative perceptions about software developers" [7]. Some developers (programmers, database developers, systems analysts) have even been unjustly characterized as "bumbling" [5]. But, in fact, most software developers have an innately high need for achievement and professional growth [8], and programmers in particular have a "...general need to create things, particularly things that other people will find useful" [3].
We have turned to the practice of developing software from the perspective of developers in order to arrive at a more insightful definition of project success since they are at the core of development work [3, 4]. (Of course, we certainly acknowledge that other project stakeholders, including project managers and customers, also have interesting and important perspectives of project success, but they are not included in this investigation; see the sidebar "How the Study was Conducted"). Essentially, software developers can make or break a software development project, as McConnell [8] suggests that people are the most important element in successful development projects because they design, create, and modify software systems. In fact, most software projects that are completed late, over-budget, and/or fail to meet customer/user requirements are actually characterized by non-technical, people-related problems experienced duringoften in the early phases ofthe development process [4, 7] Further, people also represent the largest single cost in software development [2]. Hence, it is important that all project stakeholders, particularly project managers, understand what is important to the developers' perception of project success. This will ensure that a productive and creative work environment is supported [8]; such an environment in turn can lower the overall risk associated with software development projects.
Discussion
Our model of success components is depicted in the figure here, which is based on several previous studies, including [7, 8, 10, 11]. Our analysis consists of two major categories of components: those that contribute to successful software development, and those that help define successful projects from the perspective of software developers. The first category can be thought of as drivers of success (rather than assisting in defining success). For this set of components, we asked developers to rate how significant they believe each component is in its contribution to what they consider a `successful' software project. Included components have implications for individual developers and the development team, as well as those that address the project itself (including project management, customer/users, and requirements). Indeed, some components have implications for both developers/the team and the project as a whole, such as requirements were clear and understood by the development team, and requirements were accepted by the team as being realistic and achievable. To investigate the second major category of components, developers were asked to rate how important they believe each component is, in general, to their definition of successful software project. In this case, investigated components were related either to the individual developer (related to the work they perform) or the project at large (meeting requirements, and timeliness, costliness, and ease of use of the final system).
Process-Related Components and Their Contribution To Project Success
Our 15 process success drivers are presented in Table 1, ranked by descending frequency of responses indicating high or very high importance (responses "6" or "7") in their contribution to the process of successful project development. The three highest-ranked components were as follows (shown with combined percentage of "6" and "7" responses):
- Requirements are clear and understood by the development team (90%).
- Overall, the development team is sufficiently skilled (87%).
- Customer/users and development team have a good relationship (cooperative, mutually responsive) (80%).
In general, the items depicted in Table 1 drive project success from both the perspective of the developer (an internal view) and customer/user (an external view). The highest-ranking process component was having clear and understood requirements by the development team [9], which is an indication that developers appreciate understanding the scope of their task. It also relates to the importance of managing the expectations of both developers and customer/users as early in the process as possible. Clear and understood requirements help to lessen developers' frustration at having to redo design and code parts of a poorly understood system (a measure of internal success). At the same time, such understanding of requirements, particularly early on, will help save expensive and time-consuming rework, which contributes to the timely, affordable delivery of a well-tested system that meets customer/user requirements (external objectives).
Having a sufficiently skilled development team was ranked second highest and this component has been cited as one of the "10 signs of IS project failure" [9]. It seems to support team productivity and lessen the detrimental impact of excessive training of team members. Customer/users and the development team having a good relationship was the third highest-ranked process component, and it seems to support customer/users having realistic expectations, which 97% of developers indicated that realistic expectations had at least some importance. Further, senior management and project management can, and should, play a major role in ensuring that expectations of the developing organization, as well as the customer/user, are realistic and accepted by all project stakeholders [7].
Work-Related Components and Project Success
The list of our five work-related aspects is presented in Table 2, ranked by descending frequency of responses indicating high or very high importance (responses of "6" or "7") in their contribution to developers' definition of software project success. (We investigated intrinsic components because developers are likely to place a higher value on the intrinsic value of the work itself rather than on extrinsic factors (compensation, working conditions, and appropriate technical resources) [8]. The three highest-ranked components were as follows (shown with combined percentage of "6" and "7" responses).
- You had a sense that you delivered sufficient quality (did a good job) (93%).
- You had a sense of achievement (83%).
- You were provided with enough independence and freedom (autonomy) to work creatively on a project (66%).
Similar to our investigation of process-related components, the items depicted in Table 2 help to define project success from both the perspective of the developer (an internal view) and the customer/user (an external view). These components have implications for the intrinsic motivation of developers. Further, it has been suggested that motivation has the single largest influence on productivity [1, 8]. As a result, it makes sense that these work-related components (given their relationship with productivity) have important implications for meeting such customer/user expectations of success as meeting cost, schedule, and requirements specifications. The highest-ranked work component was having a sense of delivering sufficient quality. We did not ask developers to explicitly define `quality', but, presumably, it is related to a well-tested system that meets customer/user requirements. The second highest-ranked work component, having a sense of achievement, which is an intrinsic need and, as previously mentioned, is typical of developers [8]. Being provided with enough independence/freedom (autonomy) to work creatively on a project [6] was the third highest-ranked work component. This can support the creative nature of software developers (internal component of success), which, in turn, can facilitate a motivated team. An inspired team will presumably contribute toward meeting organizational goals, including the completion of a project that is on time, within budget, and meets customer/user requirements (external components of success).
System/Project-Related Components and Project Success
System/project success included components related to either the final system or the project. Our list of nine system/project-related aspects is presented in Table 3, ranked by descending frequency of responses indicating high or very high importance (responses of "6" or "7") in their contribution to developers' definition of software project success. The three highest-ranked components were as follows (shown with combined percentage of "6" and "7" responses).
- The final system worked as intended (97%).
- Requirements of customer/users were met by the completed system (90%).
- Project was delivered when needed by customer/users (83%).
- Final system consisted of solid, thoroughly tested code (83%).
Again, as was the case with the investigated work-related components, in general, the items depicted in Table 3 help to define project success from both the perspective of the developer (an internal view), and the customer/user (an external view). For example, well-tested code presents both an internal view of success (consideration for developers), as well as external (implications for customer/users). The highest-rated components of requirements of customer/users were met by the completed system and the final system worked as intended [7] point to developer focus on user satisfaction, as does thoroughly tested code and ease of use. In fact, we see that developers tended to assign more importance to the functional aspects of the system ("requirements were met," "system worked as intended," "system consistent of solid, thoroughly tested code") than they did to the organizational/managerial aspects (completed on time, within budget, at an affordable cost). Essentially, developers indicated they value producing a quality system that meets customer/user requirements more than delivering that system on time and within budget. This is further emphasized by their ranking the delivering of a system when needed by the customer/users (not necessarily within schedule) higher (third) than delivering the system on time (according to schedule, which was ranked sixth).
Lessons Learned
Given the parade of late and expensive projects cited in both industry and academic publications, it seems beneficial for project managers and, by extension, senior management, to consider that there are important and productive measures of project success (and failure) not related to cost and schedule. Although our findings should not be generalized to the population of U.S. software developers due to our small sample size, our work does provide some insight into developers' perception of project success (for the types of projects our developers were involved with). Many of our quantitative findings seem to make intuitive sense, and largely support prior research, even if anecdotal in nature, which suggests that developers can, indeed, find value in projects that are delivered late and cost more than anticipated [5, 7, 8]. It has been suggested that many problems associated with software development relate to establishing expectations at the beginning of a project rather than events that occur during the development process [7]. Our data suggests developers also perceive these early expectations to have an important contribution to a successful development effort.
Project managers should try to support a challenging, autonomous, and creative work environment and recognize their team's need for achievement, continuous learning, and desire to do quality work (for example, deliver an easy-to-use, stabile system made up of well-tested programming code). Managers should also strive to include the development team in decisions that directly affect developers' work, including estimations of effort and scheduling, which relates to the relative amount of schedule pressure under which they work. However, this may not be possible in a situation where the organization is bidding for a software contract. Many developers indicated they partially define success in terms of not only meeting a customer/user's needs, but also "...when needed by the customer/user's (and not necessarily within schedule)."
Project managers should try to support a challenging, autonomous, and creative work environment and recognize their team's need for achievement, continuous learning, and desire to do quality work.
A logical next step in our research is to gather information via case studies that refer to completed software development projects. We could then include project context, such as "bespoke" software, customized package, in-house development, and so forth. We could also test if particular success drivers can be used to predict project success, both from an internal perspective (related to the work that developers perform) and external perspective (related to organizational outcomes, such as requirements were met, delivered on time and within budget). It would also be interesting to discover how U.S. developers' perception of project success compares to that of developers from other cultures and countries. Lastly, we are currently compiling data from software project managers in order to compare their perception of project success with that of the developers on their development teams.
References
1. Boehm, B.W. Software Engineering Economics. Prentice Hall, Englewood Cliffs, NJ, 1981.
2. Brady, S. and DeMarco, T. Management-aided software engineering. IEEE Software 11, 6 (Nov. 1994), 2532.
3. Brooks, Jr., F.P. The Mythical Man-Month (Anniversary Edition). Addison-Wesley, Reading, MA, 1995.
4. DeMarco, T. and Lister, T. Peopleware: Productive Projects and Teams (2nd Edition). Dorset House, New York, 1999.
5. Glass, R.L. Evolving a new theory of project success. Commun. ACM 42, 11 (Nov. 1999), 17.
6. Hackman, J.R. and Oldham, G.R. Work Redesign. Addison-Wesley, Reading, MA, 1980.
7. Linberg, K.R. Software developer perceptions about software project failure: A study. The Journal of Systems and Software 49, 2/3 (Dec. 30, 1999), 177192.
8. McConnell, S. Rapid Development. Microsoft Press, Redmond, WA, 1996.
9. Reel, J.S. Critical success factors in software projects. IEEE Software 16, 3 (May/June 1999), 1823.
10. Saarinen, T. An expanded instrument for evaluating information system success. Information & Management 31, 2 (Nov. 1996), 103118.
11. Stamelos, I., Angelis, L., Dimou, P. and Sakellaris, E. On the use of Bayesian belief networks for the prediction of software productivity. Information and Software Technology 45, 1 (Jan. 1, 2003), 5160.
Tables
Table 1. Process-related aspects ranked by descending combined "6" and "7" responses (percent of total responses to each question).
Table 2. Work-related aspects ranked by descending combined "6" and "7" responses (percent of total responses to each question).
Table 3. System/project-related aspects ranked by descending combined "6" and "7" responses (percent of total responses to each question).
©2006 ACM 0001-0782/06/0800 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.
No entries found