Communications of the ACM

Review articles

The ABCs of Effectiveness in The Digital Society

By Carlo Gabriel Porto Bellini
Communications of the ACM, July 2018, Vol. 61 No. 7, Pages 84-91
10.1145/3205945
Comments

The use of information and communication technologies (ICTs) by individuals is a long-time concern for researchers and practitioners. ICT use starts with the inclusion of people in the digital society and progresses toward the equalization of their capabilities and opportunities in technology-mediated information and communication processes. Approaches to inclusion and equality have become increasingly sophisticated through developments in human-centered computing and human-computer interaction that replace the old focus on people's mere access to the ICTs. At the same time, a third, more empowering moment of ICT use is attracting scholars, professionals and, hopefully, public agents—the effectiveness with which people use the technology. In this article, I discuss inclusion, equality and effectiveness under the concept of one's digital effectiveness.

Back to Top

Key Insights

Digital effectiveness manifests in three dimensions—access, cognition, and behavior. It refers to one's use of ICTs for private or professional purposes according to an arbitrarily defined effectiveness criterion. The focal point of digital effectiveness is the individual ICT user and the levels of purposeful ICT use he or she achieves. That is, the focus is on the basic building block of the digital society—the embryo of a society's digital culture and digital health. Digital effectiveness describes an individual's use of ICTs in desirable ways, regardless of whether the individual masters the ICTs or not.

This approach is an extension of the two-order digital divide perspective^4,5,6 coupled with developments in use effectiveness.^3,7,8 The rationale is as follows: (1) an individual should have proper access to the ICTs, (2) possess the cognitive potential to use them, and (3) activate the needed behaviors to operate the ICTs in practice (4) for a specific purpose (5) in reference to an effectiveness criterion (6) arbitrarily defined and measured against a stakeholder's utility function (7) that takes as input the individual's digital capabilities and limitations.

This approach therefore deals with the enablers of ICT access, the cognitive enablers of potential ICT use, and the behavioral enablers to leverage the actual benefits from ICT use. Furthermore, the three critical dimensions of access, cognition and behavior are here conceived in broad terms. Access to the ICTs is not oversimplified as the individual's socio-material setting or mere contact with technology at home, work, school, cybercafes or someone else's venue; rather, access also includes the contextual conditions of ICT use, that is, environmental ergonomics. ICT-related cognition, in turn, includes all technology-mediated information and communication mental activity reflecting formal and informal education, personal experiences, emotions, and the chain of attitudinal factors that precede actual behavior. Finally, ICT-related behavior includes the actual use of technology as a result of one's latent potential, personal deliberation, and real possibilities. In all three dimensions, the diagnostic of effectiveness depends on, first, defining who is the interested party (the main stakeholder of ICT use), and, second, the effectiveness criterion (the ICT use purpose) that serves as reference for action. Thus, it is meaningless to address digital effectiveness in theory or in practice before those two definitions are available.

As noted earlier, an individual's three dimensions of digital effectiveness often manifest in natural sequence—access first, then cognition, then behavior. But they may be also causally linked in certain circumstances, when a particular digital limitation gives rise to another limitation in the same or in a different dimension. For instance, an individual's poor ICT skills (a cognitive limitation) may lead to technophobia (a cognitive or behavioral limitation, depending on how it is defined and measured), and vice-versa. The possible causal link between limitations is one of the reasons why we measure limitations instead of capabilities to estimate one's digital effectiveness. After the measurement of limitations in a given dimension, the level of capabilities in that same dimension is the difference between the upper limit of the chosen scale (say, 100, as in a percent scale) and the computed level of limitations. Finally, the digital effectiveness value is the difference between the aggregated, normalized digital capabilities and limitations in all three dimensions of access, cognition, and behavior. This is discussed in more detail later.

The digital effectiveness approach contributes in many ways to theory and practice, such as in:

• Broadening, organizing, and unifying the complex discussion on digital divide and inequality, along with introducing technology use effectiveness as a related concern;
• Differentiating between technology mastering and use effectiveness;
• Stimulating public and social agents, organizational managers, community leaders, families, and individuals to better understand the complexity of technology use as critically dependent on proper access to technology, the development of cognitive potential to deal with technology-mediated information and communication processes, and the development of healthy behaviors toward technology;
• Identifying innate personal traits and attitudinal mechanisms that impact one's technology use effectiveness;
• Stimulating technology-related personal awareness (by doing self-evaluations about the use of ICTs), family counseling (by mentoring family members about the ICTs), organizational improvement (by rationalizing the use of ICTs at work), community empowerment (by promoting digital literacy and citizenship), efficiency and transparency in public administration (by streamlining electronic government processes), and social change (by means of user-generated content and regulation in virtual social networks).

Back to Top

Effectiveness in the Digital Age

The development of digital capabilities and the mitigation of digital limitations is a concern for individuals and groups. The digital society demands personal awareness, family counseling, community leadership, organizational vision and public policies that promote the proper access to, and the purposeful use of, the ICTs. Digital effectiveness is how I broadly refer to this desirable state of positive outcomes to be realized in the interaction of humans, information, and technology.

Bellini et al.¹ outlined a three-dimension digital limitations model expected to inform the digital divide/inequality literature in the fields of communications, information science, sociology, and public policy making. I now extend their propositions and introduce ICT use effectiveness as a related concern. Together, the digital limitations model and the concept of ICT use effectiveness form the digital effectiveness approach to the dimensions of ICT-related access, cognition, and behavior. In particular, I posit that each digital effectiveness dimension can be measured within a diglim range of values for a given ICT user in reference to an ideal parameter—an arbitrarily defined, context-specific ICT use expectation, which is set by the stakeholder who is the focal beneficiary of ICT use. As a result, anyone who expects particular outcomes from the use of ICTs (for example, an employee who performs an ICT-mediated task, or an employer who provides a new ICT infrastructure to an organizational unit) may design evaluation instruments and collect empirical data to address ICT use outcomes from a diglim-level digitally limited individual or group of individuals. This relativistic rather than optimizing view of technology use builds on developments on the interplay between cognition, affect and intentions toward the Internet;⁶ Internet access types known as motivational, material, skills, and usage access;¹³ Internet access, skills, and use;⁹ the conceptualization of use effectiveness;^3,7,8 the causally linked digital divides known as access, capability, and outcome divides;¹⁴ and the idea that ICT diffusion with social embeddedness follows the stages of access to technology, use of technology, and the impact of technology use.¹⁰

Back to Top

Limitations, Capabilities, and Effectiveness

Figure 1 shows three distinctive moments of the relationship between humans and ICTs. The moments nearly equate to how the literature evolved toward its focal interests in the last two decades. The human-ICT relationship starts with the inclusion of individuals in the digital society, progresses through equalizing their digital capabilities and opportunities, and eventually the individuals will develop their own relativistic perception of effectiveness and preferred route in the digital society. The relativistic perception is due to individuals having different ICT use purposes and possessing different personal traits and attitudinal mechanisms. As such, ICT-capable individuals reach at different, personally meaningful states of ICT use.

Figure 1. From inclusion to effectiveness in the digital society.

Figure 2 illustrates the three dimensions of one's digital effectiveness. The three dimensions are plotted in a strictly positive Cartesian space that represents an individual's digital limitations. The strictly positive assumption means that magnitudes are measured on a ratio scale (so, negative values are meaningless) and that it is axiomatically impossible to eliminate all residues of digital limitations (so, a zero is also meaningless).

Figure 2. An individual's 3D digital limitations that compromise digital effectiveness.

Digital effectiveness is then defined as the difference between digital capabilities and limitations. The proposition of measuring limitations instead of capabilities in order to draw the coordinate (a,b,c) in Figure 2 and infer about effectiveness is based on the idea that it is more efficient to identify what does not match a pattern than the opposite. This is true in a vast number of situations. For instance, in the philosophy of science, all statements are assumed to be falsifiable,¹¹ that is, there is an exception to every rule—so, we should look for the exceptions. Another example is that the process of building and using computer information systems is "never ending and error prone,"³ thus an important principle in software testing is to search for failure. Accordingly, if we focus on identifying one's digital limitations instead of capabilities, we simply need to look for any deviance from the pattern (the expected digital capability), in a management-by-exception fashion. Also, a digital limitation may be the direct cause of another limitation, so priority should be given to identifying limitations instead of capabilities.

---

Digital effectiveness manifests in three dimensions—access, cognition, and behavior.

---

The first dimension where we measure the occurrence of digital limitations refers to one's social, material and contextual barriers to properly access and use the ICTs in information and communication processes. It reflects access limitations (A_lim). Access limitations manifest through the levels of social exclusion, the lack of Internet access and desirable bandwidth, obsolete hardware and software, poorly designed human-computer interfaces and office furniture, rooms that are not noise- or smoke-free, rooms that are not clean or controlled for temperature and privacy, insufficient time to perform the tasks in the computer, and other factors.

The second dimension refers to barriers in one's neurological structure, educational background, information-processing capabilities, and hands-on experience that undermine the potential use of ICTs. It reflects cognitive limitations (C_lim). Cognitive limitations manifest through how one tries to search, select, process and apply ICT-mediated information, with origins in neuropsychological traits and mental disorders, incomplete formal education, lack of digital literacy and computer experience, poor general experience (that causes low functional variety), lack of interest in information processing and problem solving, unrealistic beliefs, wishful thinking, anxiety, pessimism, sadness, low self-confidence, overconfidence, and other factors.

The third dimension refers to barriers in one's complex intertwining of beliefs, attitudes and intentions that eventually result in negative behaviors toward the ICTs. It reflects behavioral limitations (B_lim). Behavioral limitations manifest through at least three archetypical behaviors: psychological barriers toward the ICTs that imply a pathological type of technology non-use (technophobia); the unnecessary, excessive use of technology (technoaddiction); and the use of technology in undesirable ways for the self or others (predatory technophilia). An individual's behavior is shaped by his or her neurological constitution, personal discretion in decision-making, traumas and addictions developed spontaneously or by influence of external sources. An individual can effect positive behaviors toward the ICTs (such as using them productively in electronic commerce, electronic government, online banking, distance learning, virtual social networks, and so on) or negative behaviors (such as using the ICTs for leisure during work or for work during leisure, propagating false information and computer viruses, promoting unethical behavior, using any source of information in excess so that it becomes a personal bias, giving less attention than needed to available information relative to a particular issue, and so on). Behavioral limitations reveal that ICT use effectiveness is not only dependent on technology access and cognitive potential, but also on the actual interaction of humans and computers in order to meet voluntarily espoused or externally defined utilitarian purposes.^a

However, a given behavior is technically considered a personal limitation only when that behavior does not promote the arbitrarily defined purpose of ICT use. Therefore, behaviors that could be seen as limiting one's digital effectiveness, such as non-work-related computing (also known as cyberslacking or cyberloafing) and routine-preserving behavior (also known as resistance to change), will be considered actual behavioral limitations and a threat to digital effectiveness only if they prevent one to meet the criterion of ICT use effectiveness that is defined by those who have an interest in, or who are affected by, the particular behaviors.

Table 1 synthesizes the important concepts of digital effectiveness and provides a few examples of phenomena that may be addressed with the digital effectiveness approach.

Table 1. Key concepts and examples of limitations.

It is important to note that cognitive, affective and hedonic events are here conceived as a single dimension. In fact, it is very difficult—if not impossible—to separate rational thinking from pure sentiments. Cognition and the other two dimensions are modeled here to be critical and self-explanatory: the access dimension addresses phenomena that are mostly external to the individual; the cognitive dimension addresses phenomena that occur exclusively in the individual's mind (cognition, affection, hedonism, and the stock of data, information, knowledge, and wisdom¹²); and the behavioral dimension addresses phenomena that the individual instantiates through real actions. If the digital effectiveness approach distinguished between types of mental processes, there would be overlapping areas of mental dimensions and much confusion on how to plot the Cartesian coordinate for a given mental phenomenon. The cognitive dimension thus includes all ICT-related personal evaluations (apart from the effective actions they precede) that are rational or irrational, deliberate or instinctive, conscious or unconscious, planned or emergent, evidence-based or emotional.

The three dimensions can be thus plotted like in Figure 2 to illustrate one's digital limitations after the components of each dimension are identified, measured, and normalized. Normalization is needed because the components will likely refer to very different phenomena. Also, as the effectiveness criterion is arbitrarily defined by a stakeholder in a specific situation of ICT use, the instruments to measure the components need to be customized.

The notation A_lim + B_lim + C_lim = D_lim synthesizes the assumption that one's Digital limitations are the result of computing his or her access limitations, Behavioral limitations, and Cognitive limitations in order to also estimate the complementary symptom—one's digital capabilities (Figure 3). A digital limitation is the degree to which an individual is limited in his or her capabilities to use the ICTs in reference to an arbitrarily defined, context-specific ICT use objective. And digital effectiveness is the difference between capabilities and limitations. The resulting value represents an excess of capabilities (a positive value), an excess of limitations (a negative value), the equivalence of capabilities and limitations (a zero value), or the individual's digital evolution (a variation across measurements).

Figure 3. An illustrative dimension of digital effectiveness.

Hypothetically, if we adopt a percent scale to measure each digital limitation dimension, and if diglim_δ, δ_∊{A,B,C}, is the degree of one's particular digital limitation, then his or her digital effectiveness is ((100–diglim_A)–diglim_A+(100--diglim_B) – diglim_B+(100–diglim_C)–diglim_C)/3, or (Σ_δ∊A,A,C} 100–2*diglim_δ)/3. As an illustration, if someone has an access limitation of 30% (access capability of 70%), a cognitive limitation of 50% (cognitive capability of 50%) and a behavioral limitation of 10% (behavioral capability of 90%), his or her digital effectiveness is 40%, that is, ((70-30)+(50-50)+(90-10))/3. This value will be useful if three conditions hold:

• The three dimensions are critical, that is, no dimension should be weighted for being more important than another;
• The components of all three dimensions are normalized; and
• The components selected to describe each dimension follow the 80–20 rule (Pareto principle), that is, most of the total variance of the phenomenon results from vital few components.

Measuring digital effectiveness as the difference between capabilities and limitations gives rise to interpretations. If the computed average of limitations in the three dimensions is 30% and the computed average of capabilities is 70%, then the individual's digital effectiveness is 40%—but what does this mean in practice?

So far, the best interpretation possible is given by theory-based versus heuristics-based knowledge. In most practical decisions of everyday life, we decide in favor of a given alternative on the basis of estimating its benefits and costs, irrespectively of whether we know of any supporting theory. If one says "weighing up the pros and cons, I prefer to stay in my current job," he or she is selecting an alternative solution based on an estimated difference between costs and benefits, and that difference is deemed positive and significant by the incumbent of the decision. Those two computations (the difference and the significance) are mostly based on personal experience and purpose, not on theory—that is, the decision maker arbitrarily assigns an expected utility one is able to compute based on the evidence he or she is able to collect and organize when thinking about the space of alternative solutions.

Personal determination, family and community counseling, organizational mentoring, and public policies should seek to minimize an individual's or a group's diglim_δ in order to maximize digital capabilities and, ultimately, digital effectiveness. High levels of digital effectiveness lead society to a state of digital culture, digital literacy, and, hopefully, digital health—that is, to the positive outcomes that might stem from the proper use of ICTs. Therefore, digital effectiveness is a state of desirable digital access, behavior, and cognition, or simply A + B + C = D. In other words, digital effectiveness is not a matter of mastering the ICTs, but of using them effectively in regard to use objectives that promote desired outcomes for individuals, families, communities, organizations, and the larger environment. This is critical to understand that digital effectiveness promotes positive social development. And the opposite is also true. For instance, if someone masters the technology (cognitive limitations can be assumed to be low) but uses it to harm people or other systems (behavioral limitations can be assumed to be high), we (as stakeholders) may conclude that his or her digital effectiveness is low.^b On the other hand, a very limited person in regard to ICT-related cognition who is able to deploy technology as demanded in professional routines may deliver high levels of digital effectiveness.

Back to Top

A Case

I will exemplify the main aspects of the approach with a case on electronically filing tax returns, and doing it while at work. Procrastinating tax submissions is a common phenomenon. In 2017, 50% of U.S. taxpayers did not file their tax returns until the last two weeks before Tax Day, while 40% did not until the last week; and, comparing the submissions in equivalent weeks of 2017 and 2016, the numbers were worse in 2017 in all comparisons before and after the due date.^c Contrasting the U.S. numbers with those in a country with a different economic and social reality but comparable size and population such as Brazil, the situation is similar.^d Among the U.S. taxpayers who filed their forms until one week before Tax Day, 92% did it electronically,^e while in Brazil all submissions are in electronic form since 2011. Therefore, filing tax returns is an ICT-dependent process marked by significant procrastination in practice. Although electronic tax submissions have several benefits over other forms of submission, users see the process as complex, tedious, and time consuming. However, procrastinating tax submissions or doing it incorrectly may impose important penalties on individuals, while also impeding governments to see the big picture of taxes earlier.

As many people spend hours at work each day, it is reasonable to expect non-work-related computing (NWRC) in companies when the deadline for tax submission is approaching. However, many companies have policies restricting the use of organizational resources, including the ICTs, for personal matters, whereas people use different ethical standards to guide personal behavior, sometimes indulging themselves in doing NWRC. A digital effectiveness situation thus arises, which involves at least three stakeholders—the employee (taxpayer), the employer, and the government. Table 2 synthesizes the analytical aspects for the specific case in which the employee is the focal stakeholder, that is, the one whose ICT use purpose should be met.

Table 2. A case about preparing and submitting personal tax forms using the organizational ICTs at work.

The approach promotes reflection about the ICT use situation, as there is always a need to accommodate the personal interests (the ICT use purpose) and the opportunities for action (the personally and environmentally defined limitations and capabilities). As in Table 2, enforcing or wisely relaxing the norms about ICT use in organizations is a matter of managing priorities and reducing possibly unnecessary tension, that is, recognizing the presence of different rationalities, personal needs and paths to group effectiveness. In the particular case of NWRC, it should not be always seen as detrimental to work, either because people will inevitably search for the satisfaction of pressing needs, or because judicious NWRC can be compared to a coffee break—a needed pause at work.

Back to Top

An Agenda for Research and Action

I described the key aspects of an approach to integrate digital inclusion, digital equality, and ICT use effectiveness. Someone is digitally included, equal, and effective if he or she has the desired/needed^f access to the ICTs, the desired/needed cognitive potential to use the ICTs, and the desired/needed behavior to leverage the benefits that may stem from ICT use.

Academic studies, individual and group practice, and public policy making benefit from this approach in organizing the key factors in research models, personal actions, organizational planning and social programs that target inclusion, equality, and effectiveness in the digital society. However, the digital effectiveness approach is not an operating model to be put into action directly, as it is highly abstract. Rather, it is a guide for research, practice, and policy making in positioning, measuring, and interpreting phenomena in one, two, or three interrelated dimensions and according to several assumptions. The approach is expected to have the qualities of being systemic and systematic, but it must be parameterized in each application. In particular, the components of each digital dimension and the stakeholder's effectiveness criterion should be modeled beforehand, for the measure of effectiveness to be as accurate and useful as possible. Here, I present ideas for studies and applications:

• Current studies on particular digital effectiveness topics can be positioned in the most appropriate dimension of this approach. Also, it is interesting to answer what is the main focus of already published studies in light of the approach—are current studies more interested in the components of digital limitations in specific situations, the independent variables that act on them, the intervening variables that moderate the effects, or the outcome variables of digital effectiveness? New studies might also start with the digital effectiveness approach to organize the analytical frame of reference before going to the empirical field.
• In the absence of more specific scales to measure a given limitation, each of its components might be measured according to a normalized five-point scale ranging from "fully absent" to "fully present" or according to a percent scale. Contrarily, if case-specific scales are available, Likert and percent scales might be also useful to normalize the measurements for 3D plotting.
• I proposed an interpretation for the digital effectiveness value—the arithmetic difference between capabilities and limitations. However, is it possible to define an ideal proportion or difference between limitations and capabilities in each effectiveness dimension, so that we could look directly at the proportion or the difference and straightforwardly decide if an individual is digitally effective or not? That is, is there a reference value based on statistical records of decisions made in practice and their reported outcomes on a given digital effectiveness situation?
• Another question is if the effectiveness criteria defined by different stakeholders significantly differ in most situations. For instance, what is the typical difference of ICT-related purposes between employers and employees, teachers and students, family members, or the users of a shared tool for online communications?
• Also, what is the typical positioning of group members and their clustering in the 3D digital effectiveness space, such as a work unit, a family, a class of students, or an electoral district? That is, is there a typical distance between group members in certain situations?
• Another effort would be to identify the typical individual that is most limited in each digital effectiveness dimension, given that this may reflect demographic phenomena that we are not aware of.
• The digital effectiveness approach originally describes the capabilities and limitations of a single individual. Nevertheless, it can be extended to describe how the effectiveness level of one individual impacts the effectiveness level of another individual. For instance, an individual's behavioral limitation regarding the misuse of netiquette in a virtual social network, such as when generating irrelevant information —noise—in excess, may limit the access of others to relevant information. Therefore, research on digital effectiveness might also address the interplay between the level of effectiveness of multiple individuals in a group.

The goal of the digital effectiveness approach is to promote awareness about the requisites of a purposeful, healthy digital society, starting at the individual level. Such a society has specific demands for human-centered design of technology and for how ICT innovation, diffusion, and use are conceived. In particular, I advocate that organizations, scholars and public agents should devise actions to include and equalize people in the digital society, while also assuring the positive conditions for the individuals, by themselves, to define and pursue effectiveness in the use of ICTs according to their interests and the interests of the larger environment.

Back to Top

Back to Top

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2018 ACM, Inc.

---

No entries found