Communications of the ACM
The Autonomous 'Selfie Drone'
Skypio CEO Adam Bry calls the R1 drone "an autonomous flying camera."
Credit: WeTalkUAV.com
If you're a rock climber, hiker, runner, dancer, or anyone who likes recording themselves while in motion, a personal drone companion can now do all the filming for you—completely autonomously.
Skydio, a San Francisco-based startup founded by three MIT alumni, is commercializing an autonomous video-capturing drone—dubbed by some as the "selfie drone"—that tracks and films a subject, while freely navigating any environment.
Called R1, the drone is equipped with 13 cameras that capture omnidirectional video. It launches and lands through an app — or by itself. On the app, the R1 can also be preset to certain filming and flying conditions or be controlled manually.
The concept for the R1 started taking shape almost a decade ago at MIT, where the co-founders— Adam Bry, Abraham Bacharach, and Matt Donahoe—first met and worked on advanced, prize-winning autonomous drones. Skydio launched in 2014 and is releasing the R1 to consumers this week.
"Our goal with our first product is to deliver on the promise of an autonomous flying camera that understands where you are, understands the scene around it, and can move itself to capture amazing video you wouldn't otherwise be able to get," says Bry, co-founder and CEO of Skydio.
Staying Focused
Existing drones, Bry says, generally require a human pilot. Some offer pilot-assist features that aid the human controller. But that's the equivalent of having a car with adaptive cruise control—which automatically adjusts vehicle speed to maintain a safe distance from the cars ahead, Bry says. Skydio, on the other hand, "is like a driverless car with level-four autonomy," he says, referring to the second-highest level of vehicle automation.
R1's system integrates advanced algorithm components spanning perception, planning, and control, which give it unique intelligence "that's analogous to how a person would navigate an environment," Bry says.
On the perception side, the system uses computer vision to determine the location of objects. Using a deep neural network, it compiles information on each object and identifies each individual by, say, clothing and size. "For each person it sees, it builds up a unique visual identification to tell people apart and stays focused on the right person," Bry says.
That data feeds into a motion-planning system, which pinpoints a subject's location and predicts their next move. It also recognizes maneuvering limits in one area to optimize filming. "All information is constantly traded off and balanced . . . to capture a smooth video," Bry says.
Finally, the control system takes all information to execute the drone's plan in real time. "No other system has this depth of understanding," Bry says. Others may have one or two components, "but none has a full, end-to-end, autonomous [software] stack designed and integrated together."
For users, the end result, Bry says, is a drone that's as simple to use as a camera app: "If you're comfortable taking pictures with your iPhone, you should be comfortable using R1 to capture video."
A user places the drone on the ground or in their hand, and swipes up on the Skydio app. (A manual control option is also available.) The R1 lifts off, identifies the user, and begins recording and tracking. From there, it operates completely autonomously, staying anywhere from 10 feet to 30 feet away from a subject, autonomously, or 300 feet away, manually, depending on Wi-Fi availability.
When batteries run low, the app alerts the user. Should the user not respond, the drone will find a flat place to land itself. After the flight—which can last about 16 minutes, depending on speed and use—users can store captured video or upload it to social media.
Through the app, users can also switch between several cinematic modes. For instance, with "stadium mode," for field sports, the drone stays above and moves around the action, following selected subjects. Users can also direct the drone where to fly (in front, to the side, or constantly orbiting). "These are areas we're now working on to add more capabilities," Bry says.
The lightweight drone can fit into an average backpack and runs about $2,500.
Taking Flight
Bry came to MIT in 2009, "when it was first possible to take a [hobby] airplane and put super powerful computers and sensors on it," he says.
He joined the Robust Robotics Group, led by Nick Roy, an expert in drone autonomy. There, he met Bacharach, now Skydio's chief technology officer, who that year was on a team that won the Association for Unmanned Vehicles International contest with an autonomous minihelicopter that navigated the aftermath of a mock nuclear meltdown. Donahoe was a friend and graduate student at the MIT Media Lab at the time.
In 2012, Bry and Bacharach helped develop autonomous-control algorithms that could calculate a plane's trajectory and determine its "state"—its location, physical orientation, velocity, and acceleration. In a series of test flights, a drone running their algorithms maneuvered around pillars in the parking garage under MIT's Stata Center and through the Johnson Athletic Center.
These experiences were the seeds of Skydio, Bry says: "The foundation of the [Skydio] technology, and how all the technology works and the recipe for how all of it comes together, all started at MIT."
After graduation, in 2012, Bry and Bacharach took jobs in industry, landing at Google's Project Wing delivery-drone initiative—a couple years before Roy was tapped by Google to helm the project. Seeing a need for autonomy in drones, in 2014, Bry, Bacharach, and Donahoe founded Skydio to fulfill a vision that "drones [can have] enormous potential across industries and applications," Bry says.
For the first year, the three co-founders worked out of Bacharach's dad's basement, getting "free rent in exchange for helping out with yard work," Bry says. Working with off-the-shelf hardware, the team built a "pretty ugly" prototype. "We started with a [quadcopter] frame and put a media center computer on it and a USB camera. Duct tape was holding everything together," Bry says.
But that prototype landed the startup a seed round of $3 million in 2015. Additional funding rounds over the next few years—more than $70 million in total—helped the startup hire engineers from MIT, Google, Apple, Tesla, and other top tech firms.
Over the years, the startup refined the drone and tested it in countries around the world—experimenting with high and low altitudes, heavy snow, fast winds, and extreme high and low temperatures. "We've really tried to bang on the system pretty hard to validate it," Bry says.
Athletes and Artists
Early buyers of Skydio's first product are primarily athletes and outdoor enthusiasts who record races, training, or performances. For instance, Skydio has worked with Mikel Thomas, Olympic hurdler from Trinidad and Tobago, who used the R1 to analyze his form.
Artists, however, are also interested, Bry adds: "There's a creative element to it. We've had people make music videos. It was themselves in a driveway or forest. They dance and move around and the camera will respond to them and create cool content that would otherwise be impossible to get."
In the future, Skydio hopes to find other applications, such as inspecting commercial real estate, power lines, and energy infrastructure for damage. "People have talked about using drones for these things, but they have to be manually flown and it's not scalable or reliable," Bry says. "We're going in the direction of sleek, birdlike devices that are quiet, reliable, and intelligent, and that people are comfortable using on a daily basis."
Originally published on MIT News.
Reprinted with permission of MIT News.
No entries found