Communications of the ACM
Home/Magazine Archive/September 2000 (Vol. 43, No. 9)/An Integrative Framework For IS Quality Management/Full Text
Communications of the ACM
An Integrative Framework For IS Quality Management
The importance of information technologies and the information systems function is no longer of debate among business people. The question, rather, is how an organization can take best advantage of IT in order to support its operations, add value to its products and services, and gain a competitive edge in the marketplace. To be able to perform up to such high expectations, the IS function must develop an intimate understanding of the expectations of its varied clientele. As organizations embark on their journey to be more responsive to their customers and to continuously improve the quality of their products and services, IS must do the same. Unfortunately, it seems that despite the importance of IT to the success of most organizations, the function is not proactive when it comes to actively pursuing and implementing quality principles. Surveys of IS managers [5] found that a minority of IS managers (41%) understood the basic principles of Total Quality Management (TQM), and thought they would be useful to the IS function. Even in the cases where TQM principles were understood, they often were not implemented in the IS function.
Previous research in this area is at best fragmented, with focus on only subsets of IS quality (software and data quality), and is centered around front-line (core) processes (for example, systems development). This article takes a holistic view of quality in the IS function from the perspective of an IS manager and considers issues relating to multiple stakeholder groups, product, service, and process quality. This framework represents a contribution to both the practice as well as the research of how the IS function should be managed. From the practical perspective, the framework along with the discussion of many issues relating to the implementation and management of an IS quality system can be used to introduce and pursue a philosophy of total IS quality. From the research perspective, the framework integrates and fills in gaps in the existing literature, and it proposes new research questions.
Total IS quality is a multidimensional concept. Figure 1 illustrates the six top-level dimensions of IS quality, which include:
- Infrastructure Quality: The quality of the infrastructure (hardware and enabling software) that is fielded and maintained by ISincludes, for example, the quality of the networks, and systems software.
- Software Quality: The quality of the applications software built, or maintained, or supported by IS.
- Data Quality: The quality of the data entering the various information systems.
- Information Quality: The quality of the output resulting from the information systems. In many cases, the output of one system becomes the input of another. In that respect, information quality is related to data quality.
- Administrative Quality: The quality of the management of the IS functionincludes the quality of budgeting, planning, and scheduling.
- Service Quality: The quality of the service component of the IS functionincludes the quality of customer support processes such as those related to a help desk.
These dimensions overlap considerably, and what impacts one dimension will in many cases impact other dimensions as well. For example, a decline in infrastructure quality (for example, persistent network service interruptions) is likely to cause real or perceived problems with data and service quality. And even though in the eyes of some IS customers the disctiction may be insignificant, it is nevertheless important in being able to respond appropriately. An appropriate response may include actions to address infrastructure quality (such as replacing a defective network component), data quality (such as more frequent backups), and service quality (such as improved communication on cause and resolution of the problem).
So far, research on IS quality has addressed individual components of quality and not the big picture. Pearson et al.'s framework [5] for introducing TQM into the IS function, although broad, does not provide an integrated and detailed enough perspective for managing quality in key processes and products. Also, Pitt et al. [6] present a model of IS success that is broad but incomplete. That model does not capture the variety of stakeholders affected, the interaction between IS and business processes, the administrative component of IS quality, and environmental influences on quality.
An extensive literature exists on data and information quality [8], and software quality [3]. Administrative IS quality, although it has not received as much attention on its own, is very similar to the quality of other management processes examined in the general TQM literature. Also, some parts of it, such as the quality of IS planning, have been examined [7]. Service quality also has not received as much attention in the IS literature. A paper by Pitt et al. [6], is one of a few writings in this area. As such, we would like to expand on this concept here.
The IS service component is a relative newcomer compared to the production processes associated with systems development. As IS budgets seem to continuously soar, the cost and quality of IS-related services have a great impact on an organization's cost competitiveness. This is especially true in the case of international operations that depend greatly on telecommunications, logistics, and IT. Even so, common measures of IS effectiveness focus on the products rather than the services [6]. Service quality should not be an afterthought, but a necessity if the IS organization wants customers for its next product.
In pursuing Total IS Quality, IS organizations are hampered not only by their own cultural focus on the product (at the expense of the service), but also by the fact that the majority of the existing quality management techniquesfor example, TQM or Continuous Quality Improvement (CQI)originated in manufacturing environments.
It is important to make the distinction between product and service quality for a number of reasons. Some of the product quality-improving strategies available may be inappropriate for service areas. Studies also indicate that sources of quality problems in manufacturing are different from those in service [11]. Services are fundamentally different from products in many respects. Most services are intangible (difficult to measure, test, evaluate), heterogeneous (difficult to maintain consistency from producer to producer and customer to customer), and their production and consumption are inseparable [11]. The definition of service quality, however, must be broad enough to encompass the needs of all stakeholders (more on this in the next section). Customer contact personnel and manufacturing personnel are, in effect, intermediate customers of various support services and according to Zeithaml et al. [11] "intermediate service quality problems result in problems at the consumer level."
Not all companies using TQM can claim total success. A 1992 survey by Arthur D. Little found that only a third of the responding companies felt their TQM programs were having a significant impact on their competitiveness. This is largely attributed to the fact that many firms have "concentrated all their efforts in improving their quality processes, and lost sight of the customer on the way" [10]. One of the problems is that firms tend to use short-term internal operational measures to evaluate effectiveness of their quality program, as opposed to focusing on longer term marketplace improvements. Another tendency is to focus on marginal product improvements at the expense of functions or processes about which customers care the mostservice functions. IS organizations suffer from the same limited awareness of how quality improvement activities can result in a value enhancement for their customers. As many IS functions are facing competition from outsourcers, it will be even more important for them to understand what drives customer satisfaction and loyalty.
Figure 1 recognizes the impact that the quality of the other components of a business process have on the IS component that supports that process. This is a synergistic relationship. The IS function is called to take greater responsibility not only for the systems that are fielded but also for the impact those systems have on the business processes they support [4]. If a business process is well designed, and its components are well integrated, a good quality IS will be able to maximize its impact and therefore will be judged more favorably than otherwise. It is like taking a coaching position in a team that already has some great players.
An Integrative IS Quality Framework
As Figures 1 and 2 illustrate, multiple stakeholders evaluate the quality of various IS products and processes under the influence of environmental, organizational, and departmental factors. The remainder of this section discusses these key components of the framework.
Voice of multiple stakeholders. IS stakeholders include individuals or groups concerned with the production, delivery, management, and support of IS products and services (in-process stakeholders), as well as those who consume those products and services (internal and external customers who are end-of-process stakeholders). Figures 1 and 2 illustrate the importance of recognizing the diverse group of stakeholders who eventually become the judges of IS quality.
In the past, the emphasis has been on satisfying customers internal to the organization. It is increasingly recognized that IT can be used to add value to products and services and make them more distinctive from the competing ones. The result of embedding IT in products and services, is that the final consumer becomes a user of the organization's systems and therefore an external customer of IS. Another class of external customers exists in the case of trading partners using interorganizational systems (such as EDI partners).
IS quality is the perceived quality of IS products and services in the eyes of the stakeholders. The difficulty in defining IS quality is that, with a multitude of stakeholders for any one product, there are various perceptions of quality. Different stakeholders usually have different perceptions of the importance of various IS quality dimensions and attributes. For example, systems analysts are primarily concerned with infrastructure, software, and data quality while system users are concerned with the effect of systems on a task or business process. Senior management is concerned with administrative quality of the IS planning process in terms of its alignment with organizational strategy and performance. Stakeholders of non-IS processes may indirectly evaluate IS quality. For example, someone evaluating the quality of customer service may indirectly evaluate infrastructure quality (network reliability) or information quality (quality of data available to customer service personnel).
The challenge for the IS function is to first identify all the stakeholders and their quality concerns, and then integrate all the measures and metrics from the various groups. The relative importance of each stakeholder group is likely to vary depending on culture, strategy, role of IS, politics, and other factors. Stylianou et al. [9] describe a process of gathering and integrating quality attributes from multiple stakeholders during systems development. That same methodology could be applied to define the more comprehensive IS quality advocated here.
Understanding processes. Organizations need to identify key processes relative to strategic goals and understand them in terms of stakeholders and quality attributes that are important for those processes. Each process could impact multiple IS quality dimensions. For example, the application development process could impact software, data and information quality. Another less obvious example is the impact of the budgeting process on software quality. Poor estimates during budgeting (poor administrative quality) could result in frequent project overruns, and in poor perceptions of software quality (late, expensive).
IS processes can be classified in several ways. Different ways of classifying processes help to identify important stakeholders and quality attributes for these processes. One method of classifying IS processes is based on the degree of customer contact. Processes such as end user support and training involve a high degree of customer contact. These processes contain a large service component. Other processes such as network administration and systems programming involve a relatively low degree of customer interaction. For high customer contact processes the role of the customer in specifying, designing, and evaluating the quality of the process becomes even more important than in other cases.
Another way of classifying IS processes is by recognizing that IS products and services are often part of a larger business process that is not owned by IS. The quality of the business process should integrate IS and non-IS quality attributes. It has been advocated that IS should take responsibility not only for the technical performance of the systems that it designs and operates, but also for the use of these systems and their impact on the business processes they support [4]. That being the case, it makes sense for IS to lobby for its involvement in the design of those business processes and also to advocate quality standards for them at least as rigorous as the IS quality standards.
In some cases, the IS process itself is owned or controlled by the customer. This is becoming more the case as in many organizations the IS area is becoming more decentralized with line management authority for IS assigned to the customer area, and as end-user computing becomes more common. In many such cases, it may seem that, with the decision-making authority taken away from the IS area, the responsibility for the quality of those processes should rest entirely with the customer. In reality, however, the perception remains that IS is somehow still responsible. The expectation is that IS will provide guidance or oversight to these processes. An example of such a process is a customer controlled backup process.
IS processes can also be classified into core processes (such as application development), which directly impact the quality of IS products and services, and support processes (such as budgeting), which indirectly impact the quality of IS products or services through the quality of core processes. The quality of support processes affects the quality of final products and services through the quality of core processes. The links between the quality of support processes and the quality of core processes may be difficult to articulate and quantify. However, identifying these links may prove useful in order to manage the quality of IS processes.
Influences on IS Quality
While IS quality is a general concept, there may be significant differences between organizations in terms of how quality is defined. Different organizations may emphasize different dimensions of IS quality. Environmental factors, such as industry type and the nature of competition, affect the importance of IS in the organization [2]. Managing IS quality is more complicated in IT-intensive industries such as banking because of the large number of IS-supported processes and the high degree of integration between IS and business processes.
The impact of organizational investments in IT on IS quality may vary depending on organizational factors such as culture, politics, and senior management support. Stakeholder perceptions of what constitutes acceptable quality may also vary depending on those factors. Departmental factors such as differences in leadership style, employee skills, and IT intensity also impact IS quality perceptions and management.
These environmental, organizational, and departmental factors impact the relative importance of different stakeholder groups, the weights assigned by each stakeholder group to different quality attributes, and the metrics used to measure them. As a result, deciding how to configure and manage the IS quality program requires sensitivity to these factors.
Implementing and Managing IS Quality
In instituting an IS quality program, lessons could and should be drawn from both successes and failures in quality programs implemented in other areas. Some of these experiences, such as the value of service quality, were already discussed in previous sections. Some additional issues that should be addressed when implementing an IS quality program include the following.
Customer focus. The IS department's primary focus should be on providing products and services that add value and facilitate or contribute to customers' success. Its success depends on keeping customers satisfied by delivering quality solutions, being flexible, agile, responsive, and focused on their current and future needs. Adopting such a philosophy is an important first step in any quality approach.
Process approach. The systemic associations and dependencies between related resources, activities, and outcomes need to be recognized and managed as processes. Viewing and managing the IS organization as a system of interrelated processes greatly facilitates implementing continuous improvement activities within those processes.
Leadership. Quality programs that are successful are in most cases the result of visionary leadership willing to invest energy and resources and even more importantly committed to champion the process and lead the troops in the front line. Such leadership qualities need to be cultivated, recognized, and rewarded. Top level commitment and support is important to the success of a quality program in IS as it is in any other area. Management have to be the best advocates, champions, and salespeople. They need to lead by example, that is, follow the quality principles and publicize positive results in executive management and the administrative IS processes.
Culture. Culture can greatly influence the success of a quality program. The way the entire organization perceives and deals with QI, the cultural relationship between employees and management, the role and importance of IT, and many more culture-related issues are critical to the success of an IS QI initiative. Creating a culture of treating IS as an important and integral function in organizational change improves the ability to manage the interfaces between IS and non-IS processes.
Broad participation and teamwork. In managing IS quality, it is important to assure a cooperative effort involving all stakeholders of the business processes affected. This includes providing a working environment that is supportive, inclusive, and positive for all staff members, and promoting a team attitude valuing every team member and allowing them to participate and contribute to the overall success.
Motivating the troops. Those in charge of designing and managing the quality program should not underestimate the importance of having committed and motivated personnel. Sometimes, as a result of a barrage of past initiatives that added work and resulted in little or no benefit, there is quite a bit of cynicism among the ranks. Pushing forward and expecting that the IS personnel will fall in line because of a decree from higher up could be a great folly. It is important that QI is not perceived as another bureaucratic initiative imposed from the top to squeeze employees, or another way to measure their performance. Clearly showing those affected the expected benefits of a quality program, both in terms of work satisfaction, and in terms of personal rewards, and using an open, participative approach will go a long way toward securing their commitment.
Training. Training is clearly a great contributing factor to the success of any quality program. It helps build a foundation for the operation of quality teams as well as camaraderie among the team members. Well-trained personnel are more likely to be leaders and more vigorous pursuers of QIs rather than passive followers.
Measurement and constructive feedback. After QI interventions are undertaken it is critical to measure results systematically and provide constructive feedback to facilitate continuous improvement. In other words, the quality program should be handled as a continuous operation rather than a one-time shot. Also, a mechanism should be implemented allowing stakeholders to provide suggestions for process improvement. Quality attributes and metrics for key products and processes in the IS quality program play an important role in establishing quality levels, benchmarking, and continuous improvement. Quality standards, such as ISO 9000, and evaluation frameworks, such as the Capability Maturity Model, provide several guidelines for establishing quality attributes and metrics. Organizations can use these guidelines to develop their own lists of attributes and metrics for key products and processes. Table 1 provides illustrative quality attributes and metrics for some IS processes.
Accountability for results and rewarding achievements. As a result of a quality program, employees at the lower levels are expected to take additional responsibility for the results of the processes in which they participate. While a clear system of accountability should be implemented, it is also very important to reward teams as they achieve or exceed quality goals. Employees need to be shown that the organization is committed to the quality approach by rewarding individual and team achievement. For this, an incentive system could be established geared towards teams and their individual members. People need to see the individual benefits for identifying and implementing real improvements to products, services, and processes. Rewards should also be provided for QI suggestions. Incentives should include but not be limited to financial rewards.
Self-assessment. A process of self-evaluating the status and performance of a quality program can become a very effective way to initially introduce quality in an organization, as well as to identify concrete actions needed to sustain a quality program in the long term. Such a process could provide critical feedback to the strategic planning process and also could help in setting individual and group goals.
Summary and Future Research
The potential for use of TQM techniques for effective management of the IS function has been recognized and some techniques are in use [1]. However, progress in the use of quality management ideas for IS management has been fragmented and slow.
This article has presented an integrated view of IS quality management that considers IS products and services, different stakeholder groups, the effect of IS on different organizational processes, and the effect of the organizational environment on IS quality management. This integrated view recognizes the fact that the quality of the IS function is multifaceted and judged differently by different stakeholders. Management of IS quality can therefore be difficult. Several issues for understanding and managing IS quality have been discussed. Several extensions to this research are possible. These include case studies of implementation of the framework, and studying the effectiveness of different TQM-based techniques in the context of IS organizations.
In conclusion, although implementing a comprehensive quality program in the IS area may be complicated by many factors and obstacles, it may be too great an opportunity for most organizations to forego. As business depends more and more on information technologies for day to day operation and strategic performance, having a top quality IS function may prove to be an indispensable competitive weapon.
References
1. Cortada, J.W. TQM for Information Systems Management: Quality Practices for Continuous Improvement. McGraw-Hill, NY, 1995.
2. Cash, J.I. and McLeod, P.L. Introducing IS technology in strategically dependant companies. Journal of Management Information Systems. Spring, 1995.
3. Fox, C. and Frakes, W., Ed. The quality approach: Is it delivering? Special section, Communications of the ACM 40, 6 (June 1997).
4. Markus, L.M. and Keil, M. If we built it, they will come: Designing information systems that people want to use. Sloan Management Review, Summer 1994, 1125.
5. Pearson, J.M., Crosby, L., and Orwig, R.A. A framework for the introduction of total quality management into the IS department. Journal of Computer Information Systems (Winter 199697), 2126
6. Pitt, L.F., Watson, R.T., and Kavan, C. Service quality: A measure of information system effectiveness. MIS Quarterly, (June 1995), 173185.
7. Premkumar, G., and King, W.R. Organizational characteristics and information systems planning: An empirical study. Information Systems Research 5, 2 (June 1994).
8. Strong, D.M., Lee, Y.W., and Wang, R.Y. Data quality in context. Communications of the ACM, 40, 5 (May 1997).
9. Stylianou, A.C., Kumar, R.L., and Khouja, M.J. Total Quality Management-based systems development process. Database 28, 3 (Summer 1997), 5971.
10. The cracks in quality. The Economist (April 18, 1992).
11. Zeithaml, V.A., Parasuraman, A. and Berry, L.L. Delivering Quality Service: Balancing Customer Perceptions and Expectations. The Free Press, NY, 1990.
©2000 ACM 0002-0782/00/0900 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2000 ACM, Inc.
No entries found