Toward explainable and advisable model for self‐driving cars

Abstract

Humans learn to drive through both practice and theory, for example, by studying the rules, while most self‐driving systems are limited to the former. Being able to incorporate human knowledge of typical causal driving behavior should benefit autonomous systems. We propose a new approach that learns vehicle control with the help of human advice. Specifically, our system learns to summarize its visual observations in natural language, predict an appropriate action response (eg, “I see a pedestrian crossing, so I stop”), and predict the controls, accordingly. Moreover, to enhance the interpretability of our system, we introduce a fine‐grained attention mechanism that relies on semantic segmentation and object‐centric RoI pooling. We show that our approach of training the autonomous system with human advice, grounded in a rich semantic representation, matches or outperforms prior work in terms of control prediction and explanation generation. Our approach also results in more interpretable visual explanations by visualizing object‐centric attention maps. We evaluate our approach on a novel driving dataset with ground‐truth human explanations, the Berkeley DeepDrive eXplanation (BDD‐X) dataset.

Evaluating machine-generated explanations: a “Scorecard” method for XAI measurement science

“…At Level 7, the user reciprocates by advising the program in some way (Goyal et al, 2019;Yeh et al, 2019;Kim et al, 2021). A number of XAI researchers have proposed systems that would fall at this level.…”

Explaining Deep Learning-Based Driver Models

“…Another way would be by analyzing the controller itself (introspective explanations). A related work proposed in [52] tries to generate introspective explanations by detecting the regions of the image that causally influence the network output. In this case, the images are obtained from the environment, not from the driver.…”

Interpretable Deep Tracking

“…Second, we propose how these SCMs can be integrated into a 3D detection, multi-object tracking, and motion forecasting network, similar to [10], enabling end-to-end training and interpretability. Other relevant works also apply interpretability techniques to autonomous driving, but most focus on interpretable planners and controllers [1,17], interpretable representations [3,17], post-hoc explanations [4], and advising the planner via natural language [5,6]. In contrast, our proposed architecture explains decisions of a tracker by providing interpretable SCMs as a proxy for its network's reasoning procedure.…”