Communications of the ACM

Communications of the ACM

How to Identify New High-Payoff Information Systems For the Organization

By Ken Peffers, Charles E. Gengler
Communications of the ACM, January 2003, Vol. 46 No. 1, Pages 83-88
10.1145/602421.602424
Comments

CIOs manage the intensely competitive requests originating from throughout their organizations for increasingly constrained system development resources. At the strategic level, an organization needs new systems that create competitive advantage by adding value to products, reducing costs, or opening new marketing channels [8]. At the tactical and operational levels, it faces steady demand for system change and enhancement to adapt its processes to new environments, products, and markets, as well as for improved quality, service, and productivity.

Some of the most important projects are likely to be innovative strategic-level systems. However, smaller, less-dramatic tactical projects that call for training, user support, or incremental system improvement may also represent opportunities to improve effectiveness.

There is no shortage of new IS project ideas; technology news, maintenance needs, and competitor behavior make that a certainty. Attentive managers find far more potential IS investments, many worthwhile, than can possibly be implemented. Some especially important projects are, however, likely to be hidden in this project-idea deluge. Identifying and selecting those with the greatest potential contribution to achieving organizational goals is, consequently, a seemingly intractable task [4].

Further complicating the project-evaluation process is the relationship between the organization's IS unit and its other units. For its customers throughout the organization, IS provides a variety of services, including consulting, development, operations, and maintenance. It may compete with outside vendors and user departments to provide them; consequently, the CIO can hardly afford not to provide a credible avenue for consideration of project ideas, whatever the source in the organization.

How can a CIO use knowledge about potentially strategic and nonstrategic new systems while focusing on ideas likely to be important to the organization's overall performance? We addressed this question by extending critical success factors (CSF), a widely understood concept for identifying the performance objectives to be satisfied by strategic IT investments, with concepts from marketing research for generating bottom-up ideas for new product features to create the critical success chain (CSC) method for IS planning.

Back to Top

Critical Success Factors

Proposed more than 20 years ago by John F. Rockart of MIT's Sloan School of Management [9] to help CEOs specify their information needs in strategic decision making, CSFs are the intended performance consequences and behaviors of systems with the greatest potential contribution to achieving organizational objectives, or the things that need to go right for the organization to survive and succeed [10]. They are unique to the organization, depending on its product line(s) and market positioning.

CSFs today are part of nearly every IS planning concept. Because researchers and executives consider top-down planning necessary for maintaining a strategic perspective [12], they view the CSF focus on senior managers as a strength. In recent years, however, information systems have permeated organizations at all levels, while connecting them to upstream producers, customers, and outside information. To develop a complete portfolio of important new systems, it is therefore essential to study the views of people at all organizational levels, not only those at the executive level [6].

Broad planning participation requires economical methods that encourage participants to focus on what is most important for the organization. As such participation moves farther from the executive suite, it becomes more important to understand the reasons participants prefer specific new IS features [1]. The CSF concept focuses on the key dimension of desired performance. A planning method that explicitly explains participants' reasoning would help planners better understand participant and organizational needs.

Back to Top

Using Knowledge Around the Organization

After forming a partnership in 1998 to apply marketing research methods to IS planning needs, our first result was to extend CSF to allow IS managers to take advantage of knowledge distributed around the organization, thus helping them understand why participants prefer specific IS features.

The IS partner. We adopted the "IS partner" as a metaphor to help us think about internal and external system users and managers. The objectives for the organization's IS unit are similar to those of other actors in the organization and elsewhere: provide more value for customers while capturing some of it for the organization. Thus users across the organization and beyond can be partners with IS in adding value to products while reducing costs.

Psychologists have long recognized that individuals have unique views about how things work in an organization, as well as in the wider world. These personal constructs [5], loosely aggregated across many individuals, are called culture, whether in a country or an organization, according to personal constructs theory (PCT). Systematically aggregated, they can be a source of invaluable knowledge about systems with potential strategic value for the organization. PCT can be used to extend CSF by adapting a methodology—called "laddering"—from marketing research [3].

Laddering is a practical application of PCT for modeling the value consumers place on their preferences for product features. In a laddering study, an analyst interviews consumers in order to understand the relationships among product attributes, the expected consequences of these attributes, and the relationship between these consequences and the consumers' personal values. Aggregated over many consumers, the resulting models are highly effective for designing products with more valuable features. Laddering has been used extensively in marketing research to develop features with high consumer value for such products as wireless telephones, pharmaceuticals, and banking services.

Using CSF for IS planning, a manager implicitly develops a three-element model similar to PCT. The manager assumes that if, for instance, information systems are developed with appropriate attributes, their use will result in outcomes that are observable as improved CSF performance. Better CSF performance is, in turn, required to achieve organizational goals. We have explicitly extended CSF to incorporate these implicit PCT elements, coining the term "critical success chains" (CSC) (see Figure 1). This one-to-one mapping between PCT and CSC allows us to adapt the laddering methodology as a data-collection and analysis method for CSC and apply it to IS planning.

Back to Top

CSC Method for IS Planning

The CSC method includes four steps (see Figure 2). In steps 1–3, analysts work with participants to develop graphical CSC models. In step 4, they work with IS development staff, managers, and customers to generate feasible ideas for IS projects. Here, we outline these steps and provide a concrete example in the form of a composite case study of how CSC was implemented at Wing Fat Foods (WFF); the name of the firm and specific details have been fictionalized.

Pre-study preparation. First, the study project leader (or, in a more limited study, the analyst) determines the organizational scope of the study in terms of lines of business or business units. A CSC study models the business, so limiting its scope to a single business or product line, facility, or division limits its complexity. Imagine you are that leader focusing on the supply chain, marketing channels, infrastructure, or operations. When selecting participants, review the focus you've chosen; for example, for a marketing or supply-chain focus, customers and/or suppliers should be included among the participants. Solicit participation from representative members of senior management, middle management, and professional or journeyman line employees [2], making participation as widely representative as necessary to represent those who might have knowledge about which systems are important to the organization. Although no theoretical considerations dictate minimum or maximum numbers, 20–40 participants (depending on whether they represent distinct groups) are generally sufficient and manageable. When soliciting participation, obtain project ideas from each study participant for later use as stimulating ideas in the interviewing process.

One-on-one interviews. Participants should be presented a subset (three to five) of the project idea stimuli obtained in the previous steps and asked to rank-order them in terms of importance to the organization. For a highly ranked stimuli, ask: "Why would this project be important to the organization?" Then, referring to the answer, ask: "Why is that important to the organization?" Ask a series of such questions to collect data on associated concepts, ending with the organization's goals. Finally, ask: "What about the system makes you think it would do that?" The aim is to elicit associated concepts, ending with ideas for specific system attributes. Record the responses as linked chains, obtaining several chains of concepts from each participant.

Analysis. Individual participants use unique terms to describe their opinions. Meaningful analysis requires giving similar terms the same labels for all participants. Interpret participant statements to classify them as consistent constructs across participants. A simple procedure for interpreting the statements is to read them all, find two that are similar, and give them a common label. Continue with this qualitative clustering process until proceeding further would cause the loss of substantial information.

Map the chains into a matrix and use cluster analysis to cluster them, minimizing variation in the constructs within clusters. Transform the resulting clusters into network CSC models, representing the constructs as nodes linked by the chains and arranging the nodes to minimize crossing links. The area of each node is proportional to the number of participants mentioning the related construct. The LadderMap software package, freely available from the authors (www.peffers.com), semiautomates some of the analysis and draws an initial map. The resulting models may require some adjustment to improve readability by, say, deleting redundant links and outlier concepts. CSC models aggregate constructs collected in the interviews and represent socially constructed models of relationships in an organization among proposed system attributes, CSF performance, and organization objectives.

Ideation workshops. A workshop allows technical professionals to identify feasible project ideas to accomplish objectives implicit in the CSC models. Participation is intended to include a selection of people with knowledge sufficient to understand how feasible systems could be developed in the organization, given models describing system attributes, performance, and organization objectives. The mix of participants should reflect diverse technical skills, including analysis and design, software engineering, telecommunications, and database management; participation by business analysts and external customers may also be helpful.

Explain the CSC concept to participants, presenting the CSC models. Participants use their technical expertise to identify feasible project ideas that address the relationships identified in the models. The ideas are developed to a back-of-the-envelope standard, so participants should be encouraged to identify many of them. Details can be added later for projects deemed interesting enough to pursue further. For each system, participants should be asked to label the system and describe its nature, likely architecture, likely sourcing, cost, magnitude of risk, and expected effect on the organization, as well as the resources required for its development.

A management committee can screen the resulting CSC models and project ideas to determine which are sufficiently interesting to warrant full feasibility studies and presentation to the IT coordinating committee.

Back to Top

The Case of Wing Fat Foods

WFF is a wholesaler, delivering perishable and nonperishable foodstuffs, as well as hardware, kitchenware, and household goods, to restaurants, groceries, and similar businesses along the Atlantic Coast of the U.S. It prides itself on the quality of its delivered perishable foods, claimed fresher than competitors, and on its flexible, on-demand delivery service. In an industry with a relatively low level of IS investment, WFF hopes its IS investment represents an operational edge over competitors, allowing it to continue to compete on quality and service. We persuaded WFF's CIO to let us help him use the CSC method to identify potential new IS projects.

We engaged a skilled social scientist/IS professional as an outside analyst to conduct the study. Soliciting the participation of WFF IS users, she enlisted 25, including six senior managers, 11 middle managers (four regional managers, the comptroller, the marketing manager, two shipping department managers, and three others), five journeyman employees (from delivery, shipping and receiving, accounting, and elsewhere), and three WFF customers (two restaurant managers and a supermarket representative).

While scheduling interview appointments, she also collected project ideas to serve as stimuli. Asking each participant to describe the functionality of a system that would benefit WFF, she recorded the project name, purpose, input, output, and expected benefits, as well as who would use it. She rewrote each of the descriptions into a standard 50–100-word format. Nearly every participant suggested at least one project idea. Typical of bottom-up ideas were those related directly to its author's job and, therefore, likely to be organizationally suboptimal.

She next conducted 25–50-minute interviews with each participant, showing three system descriptions suggested by other participants and asking they be rank-ordered based on organizational importance. For the highest-ranked system idea (such as one for the shipping department), she asked why the system would be important to WFF.

Participant. "Because it would help get deliveries to customers quicker after their orders are taken."¹

Analyst. "Why is it important to get deliveries to customers quicker after orders are taken?"

Participant. "Because we could deliver fresher seafood."

Analyst. "Why is it important to WFF to deliver fresher seafood?"

She continued until the participant had reached some highest-level value or objective.

She next asked, with respect to the first reason given, what it was about the shipping system that made the participant think it actually would help get deliveries to customers quicker after orders were taken.

Participant. "Because it would take less unloading time at each stop if the trucks were loaded in reverse order of the deliveries."

Analyst. "What is it about the system that would allow trucks to be loaded in reverse delivery order?"

Participant. "It could provide a delivery-ordered packing list."

The participant had arrived at a concrete feature or attribute he or she expected would be part of the project idea, so the line of questions stopped. The analyst recorded the answers as a chain (see Figure 3), including specific features of the system idea, expected performance, and related organization values or objectives. The questions were repeated for the second most highly ranked of the three project ideas; the third-ranked idea was ignored. She collected an average of about eight chains per participant, generally several chains in response to each project stimulus.

Because each participant expressed his or her ideas using unique statements, the analyst needed to cluster and relabel statements into consistent constructs across participants. In a qualitative clustering process, she and another analyst independently read through the list of participant statements, adding a common label to the two that seemed most similar. They continued to cluster the labels in this manner until continuing would have resulted in the loss of information. They compared the labels using simple correlation to ascertain that the reliability of the clustering was satisfactory, resolving differences through consensus.

---

The CSF concept focuses on the key dimension of desired performance.

---

She then mapped the constructs into a matrix, with rows representing chains and columns representing constructs; the value of a cell was 1 if the chain contained the construct, 0 otherwise. She used a popular statistical package to cluster the chains with Ward's method, a widely used hierarchical clustering method that minimizes the within-cluster variance among clusters. She examined the resulting top-10 clusters, selecting a seven-cluster solution on the basis of her judgment about the understandability of the solution.

Mapping each cluster into a CSC map, she represented the constructs as nodes and the links in the chains as lines connecting the nodes. In constructing the maps, she grouped attribute nodes to the left, CSF nodes in the center, and organizational goals to the right. She let LadderMap create an initial mapping for each cluster, then rearranged the maps somewhat to improve clarity.

The map in Figure 4 represents an organization-specific CSC model consisting of, from left to right, descriptions of desired system attributes, resulting expected performance outcomes (CSF), and associated organization goals; for example, among the model system attributes, better information and analysis for scheduling, routing, and loading trucks was viewed as affecting CSF performance, delivery speed, timeliness, and product freshness. Altered CSF performance would affect organizational goals, number of customers served, pricing, perceived quality, and margins. The number in each circle represents the number of participants mentioning the construct.

The CSC maps were used by IS professionals and non-IS customers from within WFF as a starting point for developing a portfolio of IS proposals. In a two-day group workshop at WFF, the analyst met with six professionals from WFF's IS unit, selected to represent a variety of technical specialties involved in development projects, including systems analysis, DBMS, communications, and logistics; a marketing manager and a shipping dispatcher also participated. Each was given copies of the seven CSC maps, distributed one at a time, that were explained and then discussed. The analyst asked them to think about feasible systems that would address each of the CSCs; they were told the objective of the workshop was to use their expertise to produce back-of-the-envelope proposals for feasible IS projects addressing performance in terms of the models described by the CSC maps. The projects were specified briefly in terms of project name, description, likely architecture, resources required, cost, risk, and expected effect on the organization.

Participants remarked that the richness of the information in the models helped them better target user needs, compared to the kinds of user requests they often receive. For several CSCs, technical staff and customers discussed the details of desirable IS features from a user's perspective. Workshop participants also had access to the original individual chains on which the maps were based.

The workshops yielded 14 project ideas; the one outlined in the table involves a modest decision-support system for scheduling, routing, and loading trucks for delivery. Several of the proposals were for larger projects, though most were relatively modest; three involved support activities for existing systems, including training and updated equipment and maintenance support.

Back to Top

IS Strategy

For the CIO, managers and users are partners and customers with whom the IS unit works on a range of activities. CSC focuses on the needs of these users. In other environments, CSC seems to have potential for helping executives map IS strategy; for example, it was recently used at Rutgers University to help resolve the ambiguity inherent in having multiple stakeholders with different ideas as to what the university's systems should accomplish and to develop a portfolio of applications for a satellite campus. It was also used at Digia, a Helsinki-based telecommunications R&D firm, to model customer CSCs for mobile-commerce application features and generate a portfolio of projects, some now in development; CSC was useful in modeling the value of proposed applications.

CSC supports the use of distributed knowledge for strategic IS planning; for example, it allowed WFF's analyst to use 25 bottom-up project ideas to prompt people throughout the organization to develop models associating system features with performance objectives and organizational goals. The effort resulted in a portfolio of 14 projects ideas, each related to a socially constructed model of the organization. More important, it provided a mechanism to understand why participants prefer specific system features.

Given the expense of IS planning [11], it is notable that CSC implementation is relatively economical and not disruptive to the valuable normal activities of participants within the organization or among customer and supplier groups. Requiring only limited participant interviews makes it practical to collect data from as many participants as needed.

Involving IS customers in the CSC method also helps identify important nonstrategic IS project ideas. Planners using a top-down process are unable to consider the important projects that don't address the organization's strategic needs. Planners using a bottom-up process are inundated with mostly unimportant project ideas. The CSC method produces a manageable portfolio of only the most important ones, regardless of size, each justified by a rich model of organizational relationships.

Back to Top

Back to Top

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2003 ACM, Inc.

---

No entries found