Communications of the ACM

Opinion

A Strategy for Scaling Advanced Analytics

By Mikael Berndtsson, Anna-Carin Jonsson, Magnus Carlsson, Thomas Svahn
Communications of the ACM, December 2023, Vol. 66 No. 12, Pages 29-31
10.1145/3582075
Comments

How to gain competitive advantages over competitors should be a crucial question for any organization. Organizations that have scaled advanced analytics (predictive analytics, prescriptive analytics) beyond pilot projects and characterize themselves as data-driven tend to be top performers.¹³ Furthermore, there is no shortage of examples of how advanced analytics can be applied to a given domain.

In 2022, NewVantagePartners released its 10^th annual report on data and AI leadership among Fortune 1000 organizations.¹⁴ Although progress has been made over the years to become more data-driven, there are three major concerns in the 2022 report: only 19.3% had established a data-driven culture; only 26.5% had established a data-driven organization; and the nontechnical barriers represented 91.9% of the perceived challenges to become more data-driven. In addition, the metrics are declining. One can only speculate whether more respondents in the survey or a more critical self-assessment of the organizations are the reasons for the low figures or if the explanation can be found elsewhere. In practice, it means organizations cannot use advanced analytics to its full potential.

Barriers to becoming a data-driven organization are well known and include:³ lack of understanding, lack of skills, insufficient organizational alignment, lack of management support, and lack of vision and supporting strategies. Our experience indicates organizations take shortcuts when scaling advanced analytics and have a too-strong technical focus with limited use of an overall change management process. Similarly, Davenport and Bean⁸ suggested "… firms need more concerted programs to achieve data-related cultural change," and must include more experts in organizational change and change management to overcome the barriers.¹⁴

The question is whether scaling advanced analytics is any different from scaling the usage of a new tool, business process, or workflow, for example, a new model for inpatient care flow¹⁰ or a new sales tracking system.¹ The shared findings in Ashkenas and Matta¹ and Golden-Biddle¹⁰ indicate scaling is possible as long as one does not assume everything from the initial testing of the scaled object (inpatient care flow, sales tracking system) can be replicated. Instead, the scaled object must be adapted to the new environment together with user training. Now assume initial testing of advanced analytics is done for a specific business process in sales. Following the previous scaling recommendations, the specific business process should be scaled and adapted to new environments, and users should receive training. If an organization intends to become data-driven and establish a data-driven culture in teams, the focus of the scaling should be the usage of advanced analytics within the organization, and not how advanced analytics is applied to a specific workflow or business process. Hence, previous recommendations for scaling must be adapted when scaling advanced analytics.

Given the reported problems with scaling advanced analytics, requests for more concerted programs, and increased focus on change management, what are the key elements for a proper scaling of advanced analytics?

How to start. Many organizations have started their journey to become more data-driven with pilot projects in advanced analytics.³ Our experience indicates these pilot projects tend to be fairly straightforward. One reason is they are focused on applying technical aspects to a given dataset, for example, can this data-mining algorithm give us any new insights on our sales data? In case only decision makers and data scientists are involved, the pilot project is too heavily focused on technical aspects since employees working with sales and sales data are bypassed.

Although an initial project on advanced analytics shares many barriers and practices from any type of project, there are at least three barriers that need extra attention:² data quality; business resistance; and business overconfidence. We have encountered several initial projects in advanced analytics that have been delayed due to poor data quality or where the data has been manipulated (to fit the pilot project). Several initial projects in advanced analytics intend to mimic how a middle manager makes decisions. We have seen an increased level of business resistance among middle managers that do not want to share their knowledge with a data scientist. In contrast, we have also seen evidence of business overconfidence where senior managers make statements about algorithm accuracy that is off the scale.

Regardless if organizations start with a pilot project or apply other mechanisms, for example, work discovery,¹⁰ to collect initial experience of using advanced analytics, there are some underpinning best common practices:^2,6,7 align the initial work to existing business strategies or important goals within the organization; use a cross-functional team; and document the initial work as a preparation for making a future decision if scaling should be done or not. Hence, any initial work in advanced analytics that is carried out as a big-bang project with limited or no involvement of business users is likely to act as a catalyst for business resistance.

Scaling decision. Once the initial experience of using advanced analytics has been collected, a decision about scaling is the next step. A set of framing questions can be used to make an informed decision—these framing questions are adapted from Cooley and Linn⁷:

• To what extent are the benefits of using advanced analytics observable in practice?
• To what extent is using advanced analytics aligned with existing business strategies?
• To what extent have potential business users been involved?
• What type of enablers need to be changed, for example, policy about access to data?
• What are the known technical barriers and nontechnical barriers?
• What is the business case and vision of the scaling?
• To what extent is the vision a large departure from current practices?

Scaling plan. Following best practices, an overall scaling plan should be developed, including scaling vision, scaling process, change management process, and supporting strategies. Our experience is that few organizations have an overall vision defined to become data-driven. Instead, data-driven is used as a tool to support more business-oriented visions. The scaling process should describe who is involved in the scaling process and what type of decisions and analytics are scaled first. Similarly to the initial work, a cross-functional group should be involved in the scaling process. Finally, a suitable change management process should be selected and adapted.

Strategies. In order to avoid the pitfall of mainly developing technical strategies for scaling advanced analytics, developed strategies should align with what it means to be a data-driven organization, for example, by using a sample framework for analytics.

By following the framework in the figure here, strategies developed include:

Figure. A sample framework for analytics and associated strategies.

• Business problem/question: How can the organization encourage a culture where business problems and questions that can be investigated by advanced analytics are spontaneously generated?
• Data: Existing data strategies need to be revised. Organizations that start with advanced analytics have reported that they are back to square one with data quality since old data is used in new ways or new data is collected.³ Furthermore, bigger organizations tend to have a large number of different enterprise information systems, data warehouses, and so forth, which increases the risk data is kept in silos and has definitions that are not synchronized. For these organizations, it is essential to apply master data management before applying advanced analytics spanning the data sources. Scaling analytics can expose individuals or groups unwilling to share organizational data. According to Hart and Warne,¹² the unwillingness to share data can be seen from a culture- or politics-based perspective. From a culture-based perspective, barriers can be overcome by encouraging or educating people who are unwilling to share data, whereas from a politics-based perspective, barriers can be overcome by making data sharing easier.,¹²
• Analytics capability: An organization's analytics capabilities can be divided into:
• Technology: What type of platform and tools should be used?
• Organization: How should analytical competence be organized and integrated, for example, centralized or Center of Excellence?¹¹ Who is responsible for in-house training, for example, in data literacy and tools? What type of roles need to be established, for example, chief data officer, data translators, data product manager.
• Process: How should advanced analytics be integrated and used in the decision process? What type of decision processes can be automated?
• People: How should people work with advanced analytics, and how should skills be raised? What incentives are there for using advanced analytics? Analytics career paths?
• Business insight: How are business insights documented, distributed to others, and acted upon? What type of findings and data can be shared externally?

Although most of the strategies seem obvious, organizations tend to focus on developing mostly technical strategies or are still developing them.³ For example, 53% of the respondents in the 2022 survey of Fortune 1000 organizations had developed a corporate data strategy.¹³ To avoid working in silos, developed strategies for scaling advanced analytics need to consider and adapt to existing projects and strategies in, for example, AI, master data, or cloud computing.

---

The distinction between AI and advanced analytics is sometimes blurred as they can rely on the same tools and techniques.

---

Monitor and evaluate. Progress toward the vision at the organizational level is easily detected, for example, new departments, new roles, new policies, or new staff. Assessing what happens in practice at the team level requires a deeper analysis, preferably using analytics to assess what characterizes teams that have established a data-driven culture.

The distinction between AI and advanced analytics is sometimes blurred as they can rely on the same tools and techniques. According to Cam and Chui⁵ and Davenport and Malone⁹ most AI and advanced analytics models, for example, machine learning models, are not released into production. However, organizations that are top performers in deploying AI and analytics models into production have eased the deployment by Bisson et al.,⁴ Cam and Chui,⁵ and Davenport and Malone⁹: involving stakeholders early, cross-functional teams, focus on data governance strategies, new roles (for example, product managers, translators, AI strategist) that bridges data scientists with business users, aligning with important corporate strategies, and continuous learning programs (for example, in data literacy).

The key elements for scaling advanced analytics presented here align well with what top performers in deploying AI and analytics models focus on and best practices from change management. What is unique about scaling advanced analytics? If we put aside proper use of change management and having data strategies in place, our answer would be: an increased focus on establishing new roles that bridge technical experts and business users; and reskilling employees in data literacy.

Back to Top

Back to Top

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2023 ACM, Inc.

 

---

No entries found