Communications of the ACM
Home/Magazine Archive/January 2006 (Vol. 49, No. 1)/Seven Key Interventions For Data Warehouse Success/Full Text
Communications of the ACM
Seven Key Interventions For Data Warehouse Success
No data warehouse implementation can succeed on its own. The trick is knowing when and how to intervene. Data warehouses have tremendous potential to present information. They provide the foundation for effective business intelligence solutions for companies seeking competitive advantage. While there have been notable successes [2], there have also been significant failures [4, 7]. What accounts for such conflicting results? In the 1990s, adaptive structuration theory (AST) was developed to understand conflicting results with group decision support systems [5]. This theory analyzes the technological and contextual aspects of the application of a technology, focusing on their interactions. Using AST, we examine the interaction of context and technology, and pinpoint seven key interventions specific to that interaction for data warehouse success.
We studied a large organization, in which the data warehouse implementation was successful in some units and unsuccessful in others. From interviews with users in multiple business units, both successful and unsuccessful, we derive several insights that are at odds with previous research. First, users can champion a technology just as successfully as management. Second, a wide range of data can be accessed more successfully than a narrow range (as provided in a data mart). Third, the scope and flexibility of tools offered to users should not always follow the dictum "simpler is better." While restrictive tools make obtaining information easier for many users, some users will deem the warehouse successful only if they have the more intricate unrestrictive tools required for ad hoc queries and reports. Details of these insights will be provided later.
In the organization studied the same data warehouse was found to be both a success and a failure by different units within the company. (For the purposes of this study, success was determined only by whether the warehouse was used.) This provides an opportunity for understanding data warehouses that is free of some of the confounding aspects commonly encountered in field studies. Because we studied the same technology in the same company, as opposed to results from different systems and different companies, the observed differences are not due to differences in technology or corporate culture. We analyze these differences using the theoretical framework supplied by AST: context (social systems); technological innovation (in this case, the data warehouse); and the interaction of context and technology. Our results indicate the interaction of the context with the technology is the key to understanding data warehouse success. Based on the nature of these interactions, seven interventions are identified.
Company Overview
The company studied is a large, global financial institution based in the U.S. The company is highly successful and enjoys an excellent reputation both within its industry and among its clients. Its organizational culture can be described as very conservative and resistant to change. This conservatism extends to the company's philosophy concerning technology. While the company believes that technology is important to its corporate success, its strong preference is for mature, proven technologies. The company also has a strong belief in the uniqueness of its business model, resulting in a strong bias for the in-house development of systems. If tools or solutions are purchased, they must be flexible enough to adapt to the way the company conducts business and not dictate business processes.
In 1995, the company began an in-house development effort to design and implement an enterprise-wide data warehouse. They chose to model the warehouse using Inmon's paradigm [3], which espouses a centralized database, referred to as the "data warehouse," that is used to integrate and store data extracted from multiple operational systems. This data warehouse, in turn, feeds area-specific data marts. Over the years the data warehouse has grown in size and complexity as more operational systems have been integrated into its environment. From the beginning, this development effort was very much an IT-driven initiative. The business justification for the warehouse was never fully developed, and the potential business uses of the warehouse were not carefully enumerated. The prevailing assumption of the development team was that if the warehouse was built and filled with data, then business units would find a use for it.
If there is no management champion for the warehouse, strongly supportive users of the technology can convince management of the technology's value. Both management and end users must be convinced of the technology's value if the project is to go forward.
During the fall of 2001, the company sponsored a study to evaluate its enterprise data warehouse, including how the warehouse was being used. Representatives from the business units affected by the warehouse were interviewed to determine the nature of those impacts and the degree to which each unit had been able to use the warehouse. We use the AST framework to analyze the results of those interviews.
Intervention Points
AST recognizes that both the features of a technology and the context in which it is implemented affect the use of that technology. "No matter what features are designed into a system, users mediate technological effects, adapting systems to their needs, resisting them, or refusing to use them at all" [5]. Because there is a particularly high degree of interaction between the technological dimensions and the contextual features of a data warehouse, AST is an ideal lens for this study.
Important contextual aspects are the "rules and resources actors use to generate and sustain this system" [5]. One of the frequently cited necessary contextual conditions for the successful implementation of almost any technology is a champion [1]. It is no different for data warehouses. The attitude of a unit's leader affects all the factors leading to warehouse acceptance. At the company studied, when a leader was strongly supportive (intervention point 1 in Figure 1), users were willing to pursue continuing knowledge of the technology even after introductory training. If users do not want to use the data warehouse, then it is necessary to provide additional training or motivation to change their attitudes.
While acceptance of the data warehouse generally proceeded in a top-down fashion, there was a notable success in which the push was bottom-up (intervention point 2 in Figure 1). The users championed the warehouse because they recognized the task fit and value to the organization. They convinced their unit leader, who initially had been unsupportive of the data warehouse, of the warehouse's value and thereby converted the leader into a supporter. This result is surprising in that it is counter to most prescriptions for technology adoption. It shows that not having a champion is not necessarily a death sentence for a technology. If there is no management champion for the warehouse, strongly supportive users of the technology can convince management of the technology's value. Both management and end users must be convinced of the technology's value if the project is to go forward.
AST states that the more structured the use of a technology is, the easier it will be to develop a consensus among the users about how the innovation should be used and the appropriateness of that use. Ambiguity about use of the technology erodes the users' comfort with the technology, which in turn erodes their respect for it. The third intervention point (see Figure 2) occurs with the design of the warehouse. Data warehouses are generally designed around one of two plans. Some data warehouses are designed with a set of data marts that partition the data warehouse into smaller, focused databases tailored to the information needs of a subset of users. Other data warehouses provide a single repository that gives the users a very wide range of data.
In the company studied, business units that accessed their data through data marts were usually more successful than units that accessed it through a single repository. That is, most units either requested a data mart, or simply did not use the single repository. Data marts help reduce the inherent complexity and ambiguity associated with data warehouses by providing a slice of the data that is tailored to meet the specific requirements of a business unit. However, one of the most successful uses of this data warehouse was through a single repository. That unit had a strong need for a very broad range of data as well as contextual characteristics that distinguished it from the units that used data marts. In particular, people in that unit were proactive in trying technology. As a result, they were not intimidated by technology and were generally technically knowledgeable.
The fourth intervention point (see Figure 2) is in selecting the tools that will be available to the users. Tools range from the highly restrictive, which limit the users' choices and thereby reduce ambiguity and complexity, to the highly unrestrictive, which require the user to have more expertise. Most units that were successful with data marts preferred restrictive tools for accessing the data, but the unit that was successful with the single repository rejected restrictive tools. Those users wanted the greater flexibility offered by the unrestrictive tools, and were willing to expend additional effort for more capability.
The relevance of the task to the organization, the degree to which a technology supports a task, and the degree to which users understand the task fit can influence the acceptance of the technology. This is the fifth intervention point (see Figure 3). In this study, those business units that successfully used the warehouse clearly understood the relationship between the warehouse and the business issues relevant to the unit. In other words, they saw the applicability of the data warehouse to their tasks. This was especially true of units that received data marts, which not only reduce the ambiguity of the technology, but if properly focused, make the applicability to the unit's task more obvious. Giving a unit a single repository (instead of the more narrowly focused data mart) can overload the users with information. Beyond the design of the technology, users can be supported by training on the business applicability of the warehouse. This helps them see how the warehouse can assist them in performing their jobs.
A lack of knowledge of a technology can lead to difficulties using the technology, or even abandonment of the technology; this leads to the sixth intervention point (see Figure 3). In this study, the business units that successfully used the warehouse typically had two mechanisms to acquire knowledge. First, they had a good working relationship with the warehouse development team. Because of this relationship, members were comfortable going to the warehouse development team for help with problems concerning their data warehouse applications. Conversely, those business units that were experiencing difficulty utilizing the warehouse did not draw on this resource, and generally characterized the development team as unresponsive and "difficult to deal with." The perceived availability of the data warehouse development team as a support group is an important intervention point in the success of the warehouse. If users lack the perception of support, an intervention will be needed to create a spirit of cooperation. Without that perceived support, users will lack an understanding of the purpose of the warehouse, as well as having difficulty learning how to use it.
The second mechanism successful business units had for acquiring knowledge was experts within the unit. These super users understood the data warehouse technology itself and the business issues facing the unit, at least to the degree necessary to extract relevant data from the warehouse. In other words, they understood both the business needs of the unit and the potential of the data warehouse to meet those needs. The super users were also highly skilled at using the tools. These users were recognized as authorities within the unit concerning matters related to the warehouse and played a pivotal role in disseminating knowledge. Providing the resources necessary to create the role of super users is another intervention point for management.
The relevance of the task to the organization, the degree to which a technology supports a task, and the degree to which users understand the task fit can influence the acceptance of the technology.
We have expressed the seven intervention points as sequential; however, each of these points can be addressed throughout the project.
Conclusion
Organizations are spending millions each year on data warehouse development, but the majority of the efforts fail [6], and little is understood about why these failures occur or how to prevent them. Most previous recommendations for data warehouse applications have suggested one-dimensional approaches. By considering the interaction of the contextual and technical dimensions (the human factors and the specifics of the technology design), the previous results become less contradictory. For example, conventional wisdom holds that having a management champion with a tightly focused (data mart) design and restrictive tools will lead to success.
In this case study, we observed that the reverse situation can be just as successful. If the users see the potential of the data warehouse to deliver value to the organization, they can be the champions and convince management to adopt the technology. Similarly, because of its simplicity, the data mart approach is frequently recommended as the preferred approach. Providing what the users want and need is more important. If users understand both the technology and the organization's business processes, a single data repository may actually be more satisfying for them. In the same way, too little flexibility in tools can be just as harmful as too much. The level of tool flexibility that users require for success varies based on their technical knowledge and their business needs. This article not only explains those paradoxes, but identifies the key points at which interventions may have to occur in order to achieve the level of leadership, focus, and flexibility required for data warehouse success.
References
1. Beath, C.M. Supporting the information technology champion. MIS Q. 15, 3 (Sept. 1991), 354370.
2. Cooper, B.L., Watson, H.J., Wixom, B.H., and Goodhue, D.L. Data warehousing supports corporate strategy at First American Corporation. MIS Q. 24, 4 (Dec. 2000), 547567.
3. Inmon, W.H., Imhoff, C., and Sousa, R. Corporate Information Factory. John Wiley, New York, 1997.
4. Kelly, S. Data Warehousing in Action. John Wiley, New York, 1997.
5. Poole, M.S., and DeSanctis, G. Understanding the use of group decision support systems: The theory of adaptive structuration. Reprinted in Organizations and Communications Technology, J. Fulk and C. Steinfield, Eds., Sage Publications, Newbury Park, CA, 1990.
6. Vatanasombut, B., and Gray, P. Factors for success in data warehousing: What the literature tells us. J. Data Warehousing 4, 3 (Fall 1999), 2533.
7. Watson, H.J., Gerard, J.G., Gonzalez, L.E., Haywood, M.E., and Fenton, D. Data warehousing failures: Case studies and findings. J. Data Warehousing 4, 1 (Spring 1999), 4455.
Figures
Figure 1. Intervention points during the project's initiation phase.
Figure 2. Intervention points during the design phase.
Figure 3. Intervention points during the training and support phase.
©2006 ACM 0001-0782/06/0100 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.
No entries found