Predicting Model Failure using Saliency Maps in Autonomous Driving Systems
Published in
arXiv.org(2019)External Links:
TL
TL;DR
A design to train a student model -- a failure predictor -- to predict the main model's error for input instances based on their saliency map is proposed.
Abstract
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learning systems. In this work, we propose a design to train a student model -- a failure predictor -- to predict the main model's error for input instances based on their saliency map. We implement and review the preliminary results of our failure predictor model on an autonomous vehicle steering control system as an example of safety-critical applications.
References27 items
1
Not Using the Car to See the Sidewalk — Quantifying and Controlling the Effects of Context in Classification and Segmentation
2
Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control
3
Sanity Checks for Saliency Maps
4
Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
5