Communications of the ACM
News
Autonomous Helicopters
Eugene Fratkin
There would seem to be a clear market niche for unmanned helicopters. Equipped with lightweight onboard cameras, they could serve as mapping agents or search-and-rescue "eyes" in places where using a full-sized helicopter and a human crew are life threatening or cost prohibitive. Motion-picture producers have explored the use of autonomous helicopters in filming action scenes in locations where the safety of both flight crews and movie cast members could be at risk from using larger aircraft. Humanitarian groups have considered using autonomous helicopters for land-mine detection, while public safety agencies have explored using them for inspecting bridges and other structures where human inspectors might be endangered. And they are becoming mainstays in applications such as crop dusting in Japan, where the need to fly at a low altitude and spray chemicals can be dangerous for pilots.
Academic and commercial research teams have been perfecting the capabilities of autonomous helicopters for nearly two decades, with such widespread deployments as a goal. Algorithmic and technological advances are occurring at a steady pace, but regulatory roadblocks and trade restrictions are hampering market acceptance. And, though much of the cutting-edge research in autonomous helicopters demonstrates significant crossover potential between disparate computational and scientific disciplines as well as other aviation applications, many researchers find themselves stymied by these non-technological obstacles that stem from policy concerns.
"A lot of vehicles have at least kinematics that are similar to helicopters," says Adam Coates, a Stanford University Ph.D. student who coauthored Learning for Control from Multiple Demonstrations, which won the International Conference for Machine Learning's best application paper for 2008, and describes how he and colleagues programmed an autonomous helicopter to perform complex aerobatics. "But I think the biggest hurdle is regulatory. It's virtually impossible to do real UAV [unmanned aerial vehicle] operations unless you're a defense contractor or the militaryso you have to go to a big defense contractor if you want to do real UAV research."
Regulatory hurdles vary, depending on the sovereignty involved. In the U.S., for example, the Federal Aviation Administration (FAA) has yet to issue regulations regarding the use of autonomous helicopters in public airspace. A 2008 report by the U.S. General Accountability Office (GAO) noted that unmanned aircraft, whether fixed wing or rotor powered, cannot meet the National Airspace System's safety regulations for tasks such as avoiding other aircraft. Therefore, autonomous crafts' use is limited to case-by-case approval by the FAA, and usually restricted to line-of-sight operation. In Japan, the government has placed strict trade restrictions on the Yamaha RMAX autonomous helicopter, which is regarded as the industry benchmark, to prevent it from being used for military operations by unfriendly nations.
Omead Amidi, a research faculty member at Carnegie Mellon University and CEO of SkEyes Unlimited, a Washington, PA-based firm that manufactures instruments for autonomous aircraft, concurs with Coates' observation about the dearth of regulatory infrastructure hindering wider development and deployment of the craft.
"If you have a helicopter flying over your head, it's because everything about it is regulated," Amidi says. "No such thing exists for autonomous helicopters. If you could convince me to fly one of these over the head of my daughter, OK, it's ready, but I'm not doing it now."
AI to the Forefront
Despite the regulatory issues, which the GAO estimated might take 10 years to resolve in the U.S., researchers have continued to improve autonomous helicopters' capabilities. The most advanced can take off, hover, and maintain flight autonomously through a combination of advanced sensing and navigation equipment such as laser sensors, GPS modules, inertial measurement units that contain accelerometers and gyroscopes, and communications modules that communicate with ground-based computers or human pilots when necessary. The RMAX, for example, first flew fully autonomously out of visual range in Japan in 2000, following preprogrammed instructions.
While the RMAX is well suited for commercial purposes, it is also prohibitively large and expensive for applications such as the surveillance of building interiors or for bootstrapped university research programs. A base model used by the U.S. Army for research weighs approximately 185 pounds, has a rotor diameter of three meters, and costs $86,000, while fully autonomous units, complete with navigational and control equipment, can cost $1 million.
Researchers are successfully applying disparate technologies to improve the vehicles, using much smaller and cheaper helicopters than the RMAX. For example, Coates and coauthor Pieter Abbeel, now a professor in the department of electrical engineering and computer sciences at the University of California, Berkeley, utilized artificial intelligence principles to demonstrate their assertion that an off-the-shelf expectation-maximization algorithm could result in the most advanced autonomous aerobatics yet performed, using a commercially available radio-controlled hobbyist helicopter that weighed about 10 pounds.
Coates says the Stanford project was the culmination of five years of effort, in which numerous approaches were discussed and dismissed. Andrew Ng, a professor of computer science at Stanford, who advised Coates and Abbeel in their project and was a coauthor of the Learning for Control paper, says the project successfully transferred machine learning techniques into a discipline that had hitherto been extremely labor-intensive, relying on painstaking expert modeling of likely behaviors. Ultimately, they decided to have the helicopter "watch" an expert human pilot's maneuvers via data input from onboard controls and a radio receiver that saved a copy of the human pilot's control stick positions during demonstration flights.
Human-generated mapping can cost $20,000 per square mile; an autonomous helicopter could produce the same results 10 times cheaper, says Omead Amidi.
"From those two things, you can examine state changes over time and what the pilot does, and can record a whole trajectory to build up a model," Coates says.
"Previously, the most common approach to designing controllers for autonomous aircraft, both helicopters and fixed wing, was to hire a human engineer to choose parameters for the controller," Ng says. "For example, if the helicopter is pitched forward a little more than you want, how aggressively do you want to pull back on the stick? The traditional approach was to have a person knowledgeable in aerodynamics and helicopters sit down and model that. This approach can often work, but it is a very slow design process and often doesn't perform nearly as well as modern machine learning methods."
Coates and Abbeel discovered that even the most expert human pilot's aerobatic routine contains errors (or, in the language of the problem, is suboptimal). "However, repeated expert demonstrations are often suboptimal in different ways," their Learning for Control paper noted, "suggesting that a large number of suboptimal expert demonstrations could implicitly encode the ideal trajectory the suboptimal expert is trying to demonstrate."
They discovered that merely using an arithmetic average of the states observed at any given time in the expert demonstrations would fall short of arriving at the desired trajectory, explaining that, in practice each demonstration would occur at different rates, and hence make impossible an attempt to combine states from the same time-step in each demonstration.
However, by employing the machine learning algorithmwhich includes an extended Kaiman filter and a dynamic programming algorithmthe researchers were able to infer the intended target trajectory and time alignment of all the demonstrations. And, while real-time variables such as the state of the air around the craft, rotor speed, actuator delays, and the behavior of the helicopter's onboard avionics contribute to an extremely complex environment that cannot be modeled accurately, these variables can be mitigated if the programming is able to make the helicopter fly the same trajectory each time. If so, the errors caused by these variables will tend to be the same, and therefore can be predicted more accurately.
In addition to the aerobatic results of the project, Coates says the ramifications for machine learning theory go deeper. "One of the reasons people liked our paper is that it was an off-the-shelf machine learning algorithm and we solved a strange little application nobody had thought of before," Coates notes. "People know how hard this is, and to see that AI people solved this, I think has made a big impact. We had been preaching for a while that AI is the key to solving really hard problems that aren't accessible to us when we're using lots of classical methodsand if you come up with a problem and make such large strides, it really adds some weight to the argument that AI can be real and practical with algorithms that solve really hard problems."
Smaller, Lighter, Safer
The future of autonomous helicopters might be even more profoundly affected by the march to increasingly powerful processors and smaller form factors.
"One way to avoid safety troubles is by making the helicopters smaller, so there are a lot of efforts going into miniaturizing the machines," says Eric Feron, professor of aerospace software engineering at Georgia Tech University, who studied autonomous helicopters while a graduate student at MIT. "That's where I think things are going now."
Coates says the breakthrough Stanford research was greatly facilitated by increased processor capability that allowed real-time instruction every 20th of a second, which was not possible even five years ago. Additionally, the advent of microelectromechanical systems-based sensing technology, such as gyroscopes, accelerometers, and magnetometers, is leading to increased miniaturization.
Navigationally, academic researchers are now also concentrating on developing obstacle detection technology that will allow autonomous helicopters to fly safely in urban areas teeming with tall buildings, overhead wires, and light poles. Such uses are not on the near horizon, however; the ongoing safety concerns probably point to deployment in sparsely populated areas for natural resource mapping, forest firefighting, and marine search and rescue. Human-generated mapping at quarter-meter resolution can cost $20,000 per square mile, for example, while autonomous helicopters could probably deliver the same results 10 times cheaper, says Amidi.
Georgia Tech's Feron says autonomous helicopters will continue to offer researchers an excellent platform for further research in robotics, whether the researcher is an "aeronaut" who will still be utilizing them 10 years hence, or instead testing a more universally applicable methodology on the machines, and that wider deployment will indeed follow at some point.
"The safety and reliability issues are not unworkable," Feron says. "I think it's just a matter of time."
Figures
.Figure. One of Stanford University's autonomous helicopters flying upside down in an aerobically challenging airshow. For more photos and video, visit
©2009 ACM 0001-0782/09/0600 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2009 ACM, Inc.
No entries found