Communications of the ACM
Getting Audio from Still Images, Silent Videos
The idea for Side Eye was inspired by an episode of the sci-fi show “Fringe” that saw the main characters, a team of fringe science investigators working for the FBI, extracting audio from a melted pane of glass.
Credit: Matthew Modoono/Northeastern University
A machine learning tool developed at Northeastern University can obtain audio from still images and muted videos.
Using the Side Eye tool, which leverages image stabilization technology standard in most smartphone cameras, it is possible to determine the gender of someone speaking off camera and the exact words they said.
Northeastern's Kevin Fu explained that the small springs holding a camera lens suspended in liquid experience microscopic vibrations and the light is bent almost imperceptibly when someone speaks near a camera lens.
Taking advantage of the rolling shutter method of photography used by most smartphone cameras, the researchers can extract sonic frequencies from those vibrations.
Side Eye produces muffled audio, but the use of machine learning and training on certain words and audio enables it to extract a substantial amount of information, said Fu.
From Northeastern Global News
View Full Article
Abstracts Copyright © 2023 SmithBucklin, Washington, D.C., USA
No entries found