To Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehicles

Shen, Yuan; Jiang, Shanduojiao; Yan-lin, Chen; Yang, Eileen; Jin, Xilun; Fan, Yaofu; Campbell, Katie Driggs

doi:10.48550/arxiv.2006.11684

Cited by 10 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Colley et al [3] evaluated the effects of semantic segmentation visualization technique on trust, situation awareness, and cognitive load which was later expanded into other visualization techniques in [4]. Shen et al [22] and Omeiza et al [17] discussed the circumstances under which explanations are necessary and how the content of these explanations changes in the context of autonomous driving. Du et al [6] examined explanation timing and found that explanations provided before an AV acted unexpected behaviors were associated with higher trust in and preference for AVs.…”

Section: Enhancingmentioning

confidence: 99%

Improving Explainable Object-induced Model through Uncertainty for Automated Vehicles

Ling,

Wan,

Jia

et al. 2024

Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

View full text Add to dashboard Cite

The rapid evolution of automated vehicles (AVs) has the potential to provide safer, more efficient, and comfortable travel options. However, these systems face challenges regarding reliability in complex driving scenarios. Recent explainable AV architectures neglect crucial information related to inherent uncertainties while providing explanations for actions. To overcome such challenges, our study builds upon the "object-induced" model approach that prioritizes the role of objects in scenes for decision-making and integrates uncertainty assessment into the decision-making process using an evidential deep learning paradigm with a Beta prior. Additionally, we explore several advanced training strategies guided by uncertainty, including uncertainty-guided data reweighting and augmentation. Leveraging the BDD-OIA dataset, our findings underscore that the model, through these enhancements, not only offers a clearer comprehension of AV decisions and their underlying reasoning but also surpasses existing baselines across a broad range of scenarios.

show abstract

Section: Enhancingmentioning

confidence: 99%

Improving Explainable Object-induced Model through Uncertainty for Automated Vehicles

Ling,

Wan,

Jia

et al. 2024

Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

View full text Add to dashboard Cite

show abstract

“…Some of these datasets have annotations-e.g. handcrafted explanations [55], [76], vehicle trajectories [64], human driver behaviour [77], [64] or anomaly identification with bounding boxes [58], [76]-that are helpful for posthoc driving behaviour explanation. We have categorised the type of sensors used in collecting the datasets (exteroception and proprioception), provided annotations including the type of explanation the annotations instigate.…”

Section: A Perceptionmentioning

confidence: 99%

Explanations in Autonomous Driving: A Survey

Omeiza,

Webb,

Jirotka

et al. 2021

Preprint

View full text Add to dashboard Cite

The automotive industry is seen to have witnessed an increasing level of development in the past decades; from manufacturing manually operated vehicles to manufacturing vehicles with high level of automation. With the recent developments in Artificial Intelligence (AI), automotive companies now employ high performance AI models to enable vehicles to perceive their environment and make driving decisions with little or no influence from a human. With the hope to deploy autonomous vehicles (AV) on a commercial scale, the acceptance of AV by society becomes paramount and may largely depend on their degree of transparency, trustworthiness, and compliance to regulations. The assessment of these acceptance requirements can be facilitated through the provision of explanations for AVs' behaviour. Explainability is therefore seen as an important requirement for AVs. AVs should be able to explain what they have 'seen', done and might do in environments where they operate. In this paper, we provide a comprehensive survey of the existing work in explainable autonomous driving. First, we open by providing a motivation for explanations and examining existing standards related to AVs. Second, we identify and categorise the different stakeholders involved in the development, use, and regulation of AVs and show their perceived need for explanation. Third, we provide a taxonomy of explanations and reviewed previous work on explanation in the different AV operations. Finally, we draw a close by pointing out pertinent challenges and future research directions. This survey serves to provide fundamental knowledge required of researchers who are interested in explanation in autonomous driving.

show abstract

“…O NE of the major limitations preventing a public deployment of Autonomous Vehicles (AVs) is their safety. Furthermore, Shen et al [1] emphasize how the explainability of Artificial Intelligence (AI) in AVs is necessary for trust and acceptance. Hence, the automotive industry, the academic community, and regulators are developing safety assessment procedures.…”

Section: Introductionmentioning

confidence: 99%

CoPEM: Cooperative Perception Error Models for Autonomous Driving

Piazzoni

Cherian

Vijay

et al. 2022

2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

In this paper, we introduce the notion of Cooperative Perception Error Models (coPEMs) towards achieving an effective and efficient integration of V2X solutions within a virtual test environment. We focus our analysis on the occlusion problem in the (onboard) perception of Autonomous Vehicles (AV), which can manifest as misdetection errors on the occluded objects. Cooperative perception (CP) solutions based on Vehicle-to-Everything (V2X) communications aim to avoid such issues by cooperatively leveraging additional points of view for the world around the AV. This approach usually requires many sensors, mainly cameras and LiDARs, to be deployed simultaneously in the environment either as part of the road infrastructure or on other traffic vehicles. However, implementing a large number of sensor models in a virtual simulation pipeline is often prohibitively computationally expensive. Therefore, in this paper, we rely on extending Perception Error Models (PEMs) to efficiently implement such cooperative perception solutions along with the errors and uncertainties associated with them. We demonstrate the approach by comparing the safety achievable by an AV challenged with a traffic scenario where occlusion is the primary cause of a potential collision.

show abstract

To Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehicles

Cited by 10 publications

References 24 publications

Improving Explainable Object-induced Model through Uncertainty for Automated Vehicles

Improving Explainable Object-induced Model through Uncertainty for Automated Vehicles

Explanations in Autonomous Driving: A Survey

CoPEM: Cooperative Perception Error Models for Autonomous Driving

Contact Info

Product

Resources

About