Communications of the ACM
Battery-Free Technology Brings Gesture Recognition to All Devices
The AllSee prototype, integrated here with an off-the-shelf mobile phone, enables gesture recognition while the phone is still in your pocket.
Credit: University of Washington
Mute the song playing on your smartphone in your pocket by flicking your index finger in the air, or pause your "This American Life" podcast with a small wave of the hand. This kind of gesture control for electronics could soon become an alternative to touchscreens and sensing technologies that consume a lot of power and only work when users can see their smartphones and tablets.
University of Washington computer scientists have built a low-cost gesture recognition system that runs without batteries and lets users control their electronic devices hidden from sight with simple hand movements. The prototype, called "AllSee," uses existing TV signals as both a power source and the means for detecting a user's gesture command.
"This is the first gesture recognition system that can be implemented for less than a dollar and doesn't require a battery," says Shyam Gollakota, a UW assistant professor of computer science and engineering. "You can leverage TV signals both as a source of power and as a source of gesture recognition."
The technology will be described in "Bringing Gesture Recognition To All Devices," to be presented in April at the 11th USENIX Symposium on Networked Systems Design and Implementation conference in Seattle.
The researchers built a small sensor that can be placed on an electronic device such as a smartphone. The sensor uses an ultra-low-power receiver to extract and classify gesture information from wireless transmissions around us. When a person gestures with the hand, it changes the amplitude of the wireless signals in the air. The AllSee sensors then recognize unique amplitude changes created by specific gestures.
Sensors use three to four times less power than existing gesture recognition systems by harvesting power from wireless transmissions. This allows for mobile devices to always have the gesture technology on and enabled.
Gesture recognition already is possible on some mobile devices, including the Samsung Galaxy S4 smartphone. But users have to first manually enable the feature and be able to see the device for the gesture technology to work, and if left on, the gesture system quickly drains the phone's battery. In contrast, AllSee consumes only tens of microwatts of power and can always be left on. The user could gesture at the phone in a pocket or handbag to change the volume or mute the phone without having to touch or see the phone.
This technology could allow sensors to be attached to household electronics, making it possible to interact with everyday objects using gestures and also connect them to the Internet and to each other in an "Internet of Things" world.
"Beyond mobile devices, AllSee can enable interaction with Internet of Things devices. These sensing devices are increasingly smaller electronics that can't operate with usual keypads, so gesture-based systems are ideal," says Bryce Kellogg, a UW doctoral student in electrical engineering.
The UW team tested AllSee's capabilities on smartphones and battery-free sensors using eight different hand gestures such as pushing or pulling to zoom in and out. The prototype could correctly identify the gestures more than 90 percent of the time while performed more than 2 feet away from the device.
Researchers have tested the technology for response time and whether it can distinguish between other motions and those directed at it. They found that the technology's response time is less than 80 microseconds, which is 1,000 times faster than blinking an eye.
"This enables a seamless and interactive experience for the user," says Vamsi Talla, a UW doctoral student in electrical engineering. The researchers also designed a wake-up gesture that allows the system not to confuse unintentional motions for actual gestures.
This technology builds on previous work by Gollakota on leveraging Wi-Fi signals around us for gesture recognition around the home. Prior wireless gesture recognition techniques, however, consume tens of watts of power and aren't suitable for mobile or Internet of Things devices.
The research is funded by a Google Faculty Research Award and the Washington Research Foundation.
No entries found