Communications of the ACM
Interacting With Computers Through Smell
The potential of smell to direct attention could be utilized to create an alternative to notifications currently used on computers, tablets, and mobile phones.
Credit: Vaqso
We typically interact with computers and virtual environments using vision, hearing, and touch, but there is growing interest in also incorporating smell, to replicate more closely how we interact with the real world.
Odors could help people feel more present in virtual reality settings, for example, while smell could also play a role in how we make rapid decisions or recall memories when using computers.
"The question is how can we harness the power of smell in this increasingly digital world," says Marianna Obrist, professor of multisensory interfaces at University College London in the U.K.
There are still challenges to overcome before smell can be widely used as a mode of human-computer interaction, though. A better understanding of how it could impact different aspects of behavior in virtual environments, for example, is needed.
"What we are trying to do is to understand the temporal and spatial diffusion of scent in order to really understand the effect on the user," says Obrist.
In recent work, Obrist and her colleagues investigated whether smell could help direct a person's attention in a virtual reality environment. They conducted experiments in which participants were presented with an abundance of visual stimuli, and had to pick out those that looked like mint leaves. Either sounds, smells, or both were incorporated into the virtual scene, and were emitted from different positions in the virtual environment, in order to learn whether they could help guide a person towards the mint leaf icons.
Participants were assessed based on the number of mint leaves they spotted and where they were located. The position of each person's head was tracked as they completed the task, to determine the direction they were looking and whether that matched up with the location from which a scent or sound was being emitted.
Obrist and her team found participants typically directed their attention to the source of a sound or smell. Furthermore, they seemed to explore the left side more thoroughly when a sensation was delivered from the left, compared to when it originated from the right side. The researchers think that could be because all the participants were from cultures that read from left to right. "That might have some effect which would need to be accounted for in the future," says Obrist.
They also found differences in how effectively people focused their attention when sound or smell were used on their own, compared to when both modalities were combined. "The integration of sound and smell definitely produced the best results," says Obrist.
The potential of smell to direct attention could be utilized to create an alternative to notifications currently used on computers, tablets, and mobile phones. People typically are alerted visually or with sounds when they receive a new email, phone call, or social media update, for example, but such notifications can be disruptive. "Imagine you're in a work environment, but you're constantly distracted by pings and visual notifications," says Obrist. "That takes away from your focus."
In previous work, Obrist and her colleagues found that olfactory notifications could help communicate subtleties such as the relative urgency of a message. They incorporated smell notifications into the Slack messaging app, a system used in the workplace to communicate with colleagues. In one trial, a person's contacts were matched to different scents, while other odors were associated with the degree of urgency of a notification. Participants were then asked to play a memory game as a distractor while receiving scent notifications, to see if they were able to recognize the meaning behind the smells received.
The team found that although people often did not particularly trust their noses, they were able to identify the smells and their significance quite accurately. "That means that we can use scent-meaning associations more (widely)," says Obrist. "It's the same as when you learn that red means stop, and green means go."
Still, creating a system that can effectively deliver smells is yet another challenge. For almost 20 years, Yasuyuki Yanagida, a professor in the department of information engineering at Meijo University in Japan, has been working with colleagues to develop devices that can transmit smells when people interact with computers. Their goal is to be able to deliver a scent quickly to a user's nose, which will dissipate in a short time. "For several years, we have been striving to improve precision and to bring (a smell) to a specific target," says Yanagida.
Their early devices were box-shaped, with a single opening that would emit vortex rings of scented air. The device had a number of limitations, however, like the inability to control the speed at which a smell was emitted, and the need to spend time adjusting the direction of an odor. "It was very difficult to flexibly control the characteristics of a vortex ring," says Yanagida.
Now the team is developing a compact device that uses a cluster of tiny nozzles to deliver scent, rather than a single opening. Each nozzle is connected through a tube to an air compressor and valve, so it can be controlled independently. Scented air can be delivered in different directions by varying how long nozzles are open and shifting the axis of the device. By controlling the air pressure of each valve, the speed of released air can also be adjusted to control the size of a vortex ring, and thus the distribution of a smell in a space. A variety of smells can be produced by assigning different scents to individual nozzles.
So far, Yanagida and his colleagues have built a prototype and demonstrated that it can generate vortex rings by varying the speed of air emitted from each nozzle. Rather than existing head-mounted devices that deliver a scent continuously, the Meijo University researchers are aiming to create a desktop device that can emit a series of different smells. "One vortex ring can carry one kind of smell and the next vortex ring can carry another," says Yanagida.
In a more specialized way, better smell delivery devices could be incorporated into virtual environments used for rehabilitation. Some therapists are starting to use virtual reality to help treat people with post-traumatic stress disorder, for example, by recreating a traumatic event they have witnessed as part of exposure therapy. Incorporating smells could allow people to be transported back to the trauma more vividly, which could improve the outcome of the treatment. "Smell has a strong link (to emotions), so we need to be careful about how we use it," says Obrist. "But I think it can be extremely powerful in various therapeutic areas."
Sandrine Ceurstemont is a freelance science writer based in London, U.K.
No entries found