Communications of the ACM
How to Make Self-Driving Cars Safer on Roads
It's a big question for many people in traffic-dense cities: When will self-driving cars arrive? After a series of high-profile accidents in the United States, safety issues could bring the autonomous dream to a screeching halt.
At the University of Southern California, researchers have tackled a long-standing problem for autonomous vehicle developers: testing the perception algorithms of the system which allow the self-driving car to "understand" what it "sees."
The researchers describe their work in "Specifying and Evaluating Quality Metrics for Vision-based Perception Systems," presented at DATE 2019, the Design, Automation and Test in Europe conference.
Working with researchers from Arizona State University, the USC team's new mathematical method is able to identify anomalies or bugs in the system before the car hits the road.
Perception algorithms are based on convolutional neural networks, powered by machine learning, a type of deep learning. These algorithms are notoriously difficult to test, as it's not fully understood how they make their predictions. This can lead to devastating consequences in safety-critical systems like autonomous vehicles.
"Making perception algorithms robust is one of the foremost challenges for autonomous systems," says Anand Balakrishnan, the DATE 2019 study's lead author a computer science Ph.D. student at USC. "Using this method, developers can narrow in on errors in the perception algorithms much faster and use this information to further train the system. The same way cars have to go through crash tests to ensure safety, this method offers a pre-emptive test to catch errors in autonomous systems," he says.
Errant Learning
Typically autonomous vehicles "learn" about the world via machine learning systems, which are fed huge datasets of road images before they can identify objects on their own.
But the system can go wrong. In the case of a fatal accident between a self-driving car and a pedestrian in Arizona in March 2018, the software classified the pedestrian as a "false positive" and decided it didn't need to stop.
"We thought, clearly there is some issue with the way this perception algorithm has been trained," says study co-author Jyo Deshmukh, a USC computer science professor and former research and development engineer for Toyota, specializing in autonomous vehicle safety.
"When a human being perceives a video, there are certain assumptions about persistence that we implicitly use: if we see a car within a video frame, we expect to see a car at a nearby location in the next video frame. This is one of several 'sanity conditions' that we want the perception algorithm to satisfy before deployment," Deshmukh says.
For example, an object cannot appear and disappear from one frame to the next. If it does, it violates a "sanity condition," or basic law of physics, which suggests there is a bug in the perception system.
Deshmukh and his Ph.D. student Balakrishnan, along with USC Ph.D. student Xin Qin and master's student Aniruddh Puranic, teamed up with Arizona State University researchers Adel Dokhanchi, Heni Ben Amor, and Georgios Fainekos to investigate the problem.
The team formulated a new mathematical logic, called Timed Quality Temporal Logic, and used it to test two popular machine-learning tools—Squeeze Det and YOLO—using raw video datasets of driving scenes.
The logic successfully honed in on instances of the machine learning tools violating "sanity conditions" across multiple frames in a video. Most commonly, the machine learning systems failed to detect an object or misclassified an object.
For instance, in one example, the system failed to recognize a cyclist from the back, when the bike's tire looked like a thin vertical line. Instead, it misclassified the cyclist as a pedestrian. In this case, the system might fail to correctly anticipate the cyclist's next move, which could lead to an accident.
Phantom objects—where the system perceives an object when there is none—were also common. This could cause the car to mistakenly slam on the breaks—another potentially dangerous move.
The team's method could be used to identify anomalies or bugs in the perception algorithm before deployment on the road and allow developers to pinpoint specific problems.
The idea is to catch issues with perception algorithms in virtual testing, making the algorithms safer and more reliable. Crucially, because the method relies on a library of "sanity conditions," there is no need for humans to label objects in the test dataset—a time-consuming and often-flawed process.
In the future, the team hopes to incorporate the logic to retrain the perception algorithms when it finds an error. It could also be extended to real-time use, while a car is driving, as a real-time safety monitor.
No entries found