Communications of the ACM
Introduction
Although the trend to use more computing devices may provide an opportunity for increased productivity, such benefit comes at a cost. That cost is the requirement to be availableat any time or placein order to swiftly adapt to changes in our information environment. Rather than mitigate this cost, our computing devices currently exacerbate it. This is because their user interfaces have not fundamentally changed in 20 years: each device acts in isolation, as if it were the user's only device. As a consequence, devices bombard users with requests for attention, regardless of the cost of their interruptions. At the same time, computers are increasingly enveloping our senses by becoming part of our physical environment. Although ubiquitous wall-sized displays may offer relief from the tunnel vision caused by our desktop screens, this technology brings with it the capacity to further increase our information load. Consequently, growing demands on users' attention have become a critical usability issue in computing.
Researchers are becoming aware of the fact that user attention is a limited resource that must be conserved. User interface designers and engineers are beginning to design computing devices that negotiate rather than impose the volume and timing of their communications with the user. Devices are augmented such that they become aware of competing requests for their user's attention. Information systems are being developed that weigh the importance of the information they supply with the estimated priorities in user activity. Displays are created that track the user's focus of attention and adjust their renderings to provide peripheral context in support of focused activity. This area of work has created new interaction and visualization styles that aim to focusrather than distractthe user. Computing interfaces that are sensitive to the user's attention are called Attentive User Interfaces (AUIs); the user interface paradigm that is the topic of this special section.
Devices bombard users with requests for attention, regardless of the cost of their interruptions.
AUIs follow the general metaphor of a well-tuned modern traffic light system. Modern systems use sensors in the road to determine where users will go nexttheir future focus. They employ statistical models of traffic volume to determine the priority of user requests for intersection space. They use peripheral displaystraffic lightsto negotiate turn-taking activities of users sharing the limited real estate of an intersection. Likewise, AUIs may measure and model the focus and priorities of their user's attention. They structure their communication such that the limited resource of user attention is allocated optimally across the user's tasks.
AUIs use a range of novel input, interpretation, and output techniques to implement the examples mentioned here. To determine what task, device, or person a user is currently attending to, AUIs employ extra channels of input that measure overt characteristics of user attention (such as user presence, proximity, orientation, speech activity, or gaze). Based on such input, AUIs infer knowledge about the priorities in the tasks that govern their user's attention. By statistically modeling attention and other interactive behaviors of users, AUIs may establish the urgency and relevance of information or actions they offer in the context of current activity. Such interfaces with a deep understanding of user attention have been referred to as Attentional User Interfaces. We note that models of the attentive behavior of users need not be fully predictive. This is because AUIs are able to gracefully negotiate the amount of attention they receive from the user using a process similar to turn-taking behavior in conversations. Instead of immediately taking the foregroundinterrupting the ongoing activity of the userAUIs can progressively signal their requests for attention. Initially this may happen through a channel peripheral to the user's activity. AUIs may then wait for user acknowledgmentprovided through an attentive input devicebefore they take the foreground. To achieve such behavior during mediated communications, AUIs may relay information about the attention of their users to remote persons or devices. Such interfaces are known as "Attention-based" or "Awareness" systems.
Finally, AUIs can also augment the user's capacity for attention. Analogous to the Cocktail Party Effect, which allows people to focus on one among several speakers during noisy meetings, AUIs can enhance information presented in areas where users focus their attention, while they attenuate peripheral detail. The accompanying figure summarizes the features noted here, and compares them to interaction techniques currently used in GUIs.
The focus on design for user attention is what distinguishes the AUI paradigm from other, related areas of research in human-computer interaction. For example, Perceptual User Interfaces aim to optimize human-computer interactions by employing users' verbal and nonverbal communication channels. AUIs employ such perceptual information with a specific intent to identify or optimize user attention. Similarly, Context-Aware Interfaces use the context in which interactions take place to provide more relevant services to the user. However, Context-Aware Interfaces are attentive only when that context is governed by user attention. The AUI paradigm also has roots in a history of research on human attention, eye tracking, nonverbal communication, conversational interfaces, user modeling, and awareness in collaborative systems. The phrase "attentive user interface" is used in this section as an umbrella term that includes attentional and attention-based systems, as well as what have been called "attentive systems," which observe user activity to anticipate user needs.
As with every emerging paradigm, AUIs bring about new challenges for researchers. The articles in this section present some of the first milestones toward Attentive User Interfaces. Authors contributing to the section include some of the world experts on this topic who share their views and present their prototypes. Zhai lays the foundation by taking an in-depth look at IBM's work on eye trackingthe most relevant attentive input technique. He discusses how implicit measures of the user's visual attention may combine with manual action to optimize GUI interactions. Shell et al. discuss work at Queen's University and MIT toward AUIs that employ subtle turn-taking techniques to optimize user interactions with groups of computing devices. Maglio and Campbell describe their research on attentive agents, information systems that track user interest and render suggestions using peripheral ticker-tape displays. Horvitz et al. review principles for sensing and reasoning about a user's attention with Bayesian models, and for using these inferences to identify ideal decisions in messaging, interaction, and computation. Baudisch et al. take a look at advances in attentive display technologies, visualization systems that increase or decrease information density depending on the focus of user attention. McCrickard and Chewar close this section by proposing a qualitative framework for evaluating AUI designs.
Figures
Figure. This diagram shows how AUI interaction styles relate to those found in traditional GUIs. AUIs sense whether they are in the foreground of user attention by observing the presence, activity, and eye-gaze behavior of the user. In AUIs, menus and alerts are supplanted by a graceful negotiation of turns with the user through peripheral channels. Foreground activities are distinguished from background activities using seamless transitions in the density of the information rendered, rather than by a discrete windowing system.
©2003 ACM 0002-0782/03/0300 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2003 ACM, Inc.