Communications of the ACM
Robots Read Feelings With Speed and Accuracy
Woody, a social robot, reads the emotions on human faces based on algorithms developed by researchers at Case Western Reserve University.
Credit: Case Western Reserve University
Robots are getting smarter—and faster—at knowing what humans are feeling and thinking just by "looking" into their faces, a development that might one day allow more emotionally perceptive machines to detect changes in a person's health or mental state.
Researchers at Case Western Reserve University say they're improving the artificial intelligence (AI) now powering interactive video games and which will soon enhance the next generation of personalized robots likely to coexist alongside humans.
And the Case Western Reserve robots are doing it in real time.
New machines developed by Kiju Lee, the Nord Distinguished Assistant Professor in mechanical and aerospace engineering at the Case School of Engineering, and graduate student Xiao Liu, are correctly identifying human emotions from facial expressions 98 percent of the time—almost instantly. Previous results from other researchers had achieved similar results, but the robots often responded too slowly.
"Even a three-second pause can be awkward," Lee says. "It's hard enough for humans—and even harder for robots—to figure out what someone feels based solely on their facial expressions or body language. All of layers and layers of technology—including video capture—to do this also unfortunately slows down the response."
Lee and Liu accelerated the response time by combining two pre-processing video filters to another pair of existing programs to help their robot classify emotions based on more than 3,500 variations in human facial expression.
But that's hardly the extent of facial variation: Humans can register more than 10,000 expressions, and each has a unique way of revealing many of those emotions, Lee says.
But "deep-learning" computers can process vast amounts of information once those data are entered into the software and classified.
Thankfully, the most common expressive features among humans are easily divided into seven emotions: neutral, happiness, anger, sadness, disgust, surprise, and fear—even accounting for variations among different backgrounds and cultures.
Applications now and future
Recent work by Lee and Liu could lead to a host of applications when combined with advances by dozens of other researchers in the AI field, Lee says. Lee and Liu describe their work in "Optimized Facial Emotion Recognition Technique for Assessing User Experience," presented at the 2018 IEEE Games, Entertainment, and Media Conference.
The two are also now working on another machine-learning based approach for facial emotion recognition, which so far has achieved over 99-percent of accuracy with even higher computational efficiency.
Someday, a personal robot may be able to accurately notice significant changes in a person through daily interaction—even to the point of detecting early signs of depression, for example.
"The robot could be programmed to catch it early and help with simple interventions, like music and video, for people in need of social therapies," Lee says. "This could be very helpful for older adults who might be suffering from depression or personality changes associated with aging."
Lee is planning to explore the potential use of social robots for social and emotional intervention in older adults through collaboration with senior care facility Ohio Living Breckenridge Village. Senior residents there are expected to interact with a user-friendly, socially interactive robot and help test accuracy and reliability of the embedded algorithms.
Another future possibility: A social robot that learns the more-subtle facial changes in someone on the autism spectrum—and which helps "teach" humans to accurately recognize emotions in each other.
"These social robots will take some time to catch on in the U.S.," Lee says. "But in places like Japan, where there is a strong culture around robots, this is already beginning to happen. In any case, our future will be side-by-side with emotionally intelligent robots."
No entries found