Context-based image explanations for deep neural networks

Anjomshoae, Sule; Omeiza, Daniel; Jiang, Lili

doi:10.1016/j.imavis.2021.104310

Cited by 13 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The articles with a legal background (circa 2%) cover algorithmic transparency in the European General Data Protection Regulation (GDPR) [24], the applications of XAI on legal text [69], the explanation techniques in law and their applications for machine learning models [70]. In the field of robotics (circa 2%), the considered articles address the subject of explainable reinforcement learning [71,72] as well as categorization of explanatory capabilities and requirements [51]. Examples of other application fields include autonomous driving [50], communication systems and networking [49,73], education [74,75], and social sciences [11].…”

Section: Methodsmentioning

confidence: 99%

XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

Clement¹,

Kemmerzell²,

Abdelaal

et al. 2023

MAKE

View full text Add to dashboard Cite

Currently, explainability represents a major barrier that Artificial Intelligence (AI) is facing in regard to its practical implementation in various application domains. To combat the lack of understanding of AI-based systems, Explainable AI (XAI) aims to make black-box AI models more transparent and comprehensible for humans. Fortunately, plenty of XAI methods have been introduced to tackle the explainability problem from different perspectives. However, due to the vast search space, it is challenging for ML practitioners and data scientists to start with the development of XAI software and to optimally select the most suitable XAI methods. To tackle this challenge, we introduce XAIR, a novel systematic metareview of the most promising XAI methods and tools. XAIR differentiates itself from existing reviews by aligning its results to the five steps of the software development process, including requirement analysis, design, implementation, evaluation, and deployment. Through this mapping, we aim to create a better understanding of the individual steps of developing XAI software and to foster the creation of real-world AI applications that incorporate explainability. Finally, we conclude with highlighting new directions for future research.

show abstract

Section: Methodsmentioning

confidence: 99%

XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

Clement¹,

Kemmerzell²,

Abdelaal

et al. 2023

MAKE

View full text Add to dashboard Cite

show abstract

“…They further mapped these objects to TF-IDF feature vectors and utilized them for training a scene classifier to predict scene categories. As for the explainability aspect of scene classification, the work of Anjomshoae et al [51] is the most related to our approach in providing text-based explanations by using local information (i.e., objects). It generated textual explanations by calculating the contextual importance of each semantic category in the scene by masking that particular segment, and determining its effect on the prediction.…”

Section: Scene Classificationmentioning

confidence: 99%

Tbexplain: A Text-Based Explanation Method for Scene Classification Models with the Statistical Prediction Correction

Aminimehr

Khani

Amirali

et al. 2023

Preprint

View full text Add to dashboard Cite

“…CI expresses the importance of the different feature attributes for a prediction. Apart from being important, we want to know the extent to which the attributes of the different input features are favourable (or not) for a prediction, this is referred to as contextual utility [25].…”

Section: Explanation Generation Approachmentioning

confidence: 99%

From Spoken Thoughts to Automated Driving Commentary: Predicting and Explaining Intelligent Vehicles' Actions

Omeiza,

Anjomshoae,

Webb

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Commentary driving is a technique in which drivers verbalise their observations, assessments and intentions. By speaking out their thoughts, both learning and expert drivers are able to create a better understanding and awareness of their surroundings. In the intelligent vehicle context, automated driving commentary can provide intelligible explanations about driving actions, and thereby assist a driver or an end-user during driving operations in challenging and safety-critical scenarios. In this paper, we conducted a field study in which we deployed a research vehicle in an urban environment to obtain data. While collecting sensor data of the vehicle's surroundings, we obtained driving commentary from a driving instructor using the think-aloud protocol. We analysed the driving commentary and uncovered an explanation style; the driver first announces his observations, announces his plans, and then makes general remarks. He also made counterfactual comments. We successfully demonstrated how factual and counterfactual natural language explanations that follow this style could be automatically generated using a simple tree-based approach. Generated explanations for longitudinal actions (e.g., stop and move) were deemed more intelligible and plausible by human judges compared to lateral actions, such as lane changes. We discussed how our approach can be built on in the future to realise more robust and effective explainability for driver assistance as well as partial and conditional automation of driving functions.

show abstract

Context-based image explanations for deep neural networks

Cited by 13 publications

References 10 publications

XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

Tbexplain: A Text-Based Explanation Method for Scene Classification Models with the Statistical Prediction Correction

From Spoken Thoughts to Automated Driving Commentary: Predicting and Explaining Intelligent Vehicles' Actions

Contact Info

Product

Resources

About