Fault Diagnosis for the Perception Software in Highly Automated Vehicles and Reference Generation Based on World Model

Abstract

In a highly automated vehicle (HAV), the perception system performs critical sensing tasks such as object detection, scene understanding, etc. The perception software is based on complex intelligent algorithms and subjected to different failures such as missing object detection and false classification. It is a challenge to detect and identify these faults in the run time due to lack of ground truth and performance metrics. In this study, we introduce a method of perception software diagnostics, which is a generic method applicable to any state-of-the-art object detection models. In this method, perception results are compared with references generated with different sources, including pre-determined ground truth. The inconsistency between the perception results and the references, together with other performance metrics such as spatial variance, is used by a diagnostic logic to determine the fault type. Moreover, a method to generate references from a world model is presented. With validation, it is shown that the diagnostic algorithm provides a good performance in fault detection and isolation. This study enables fault mitigation in the run time and supports patch development in the development time.