Explainable AI for Understanding Decisions and Data-Driven Optimization of the Choquet Integral

Murray, Bryce; Islam, Muhammad Aminul; Pinar, Anthony J.; Havens, Timothy C.; Anderson, Derek T.; Scott, Grant

doi:10.1109/fuzz-ieee.2018.8491501

Cited by 13 publications

(14 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, a benefit of designing an explicit neural fusion network is highlighted. In [21], we established ChI indices for XAI. The reader can refer to [21] for full mathematical explanation.…”

Section: Xai For the Choquet Integralmentioning

confidence: 99%

See 1 more Smart Citation

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Islam,

Anderson,

Pinar

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multi-layer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic gradient descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables eXplainable AI (XAI). Synthetic validation experiments are provided and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy and our previously established XAI indices shed light on the quality of our data, model, and its decisions.

show abstract

Section: Xai For the Choquet Integralmentioning

confidence: 99%

“…In [21], we established ChI indices for XAI. The reader can refer to [21] for full mathematical explanation. Due to manuscript length, we are only able to summarize the indices.…”

Section: Xai For the Choquet Integralmentioning

confidence: 99%

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Islam,

Anderson,

Pinar

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since the FM has many parameters-2 N of them-XAI requires techniques to highlight and summarize relevant information for human consumption. In [93,94], we showed that existing methods make the assumption that a model is fully observed, which is not a reality in our data-driven machine learning era. This results in distorted answers, i.e., results computed using variables whose information are not supported by data.…”

Section: Information Indicesmentioning

confidence: 99%

“…While we focused on the Shapley and interaction index, the underlying formula and processes can be extended to other information indices 5. In[93,94], we produced a "truth" degree that measures how well all variables in a single ChI instance were approximated from data 6. Refer to[93,94] for full details about how to interpret Figure6.1.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Explainable data fusion

Murray¹

View full text Add to dashboard Cite

The recent resurgence of Artificial Intelligence (AI), specifically in the context of applications like healthcare, security and defense, IoT, and other areas that have a big impact on human life, has led to a demand for eXplainable AI (XAI). The production of explanations is argued to be a key aspect of achieving goals like trustworthiness and transparent versus opaque AI. XAI is also of fundamental academic interest with respect to helping us identifying weaknesses in the pursuit of making better AI. Herein, I focus on one piece of the AI puzzle, information fusion. In this work, I propose XAI fusion indices, linguistic summaries (aka textual explanations) of these indices, and local explanations for the fuzzy integral. However, a limitation of these indices is its tailored to highly educated fusion experts, and it is not clear what to do with these explanations. Herein, I extend the introduced indices to actionable explanations, which are demonstrated in the context of two case studies; multi-source fusion and deep learning for remote sensing. This work ultimately shows what XAI for fusion is and how to create actionable insights.

show abstract

Information Fusion-2-Text: Explainable Aggregation via Linguistic Protoforms

Murray

Anderson

Havens

et al. 2020

Information Processing and Management of Uncertainty in Knowledge-Based Systems

Self Cite

View full text Add to dashboard Cite

Recent advancements and applications in artificial intelligence (AI) and machine learning (ML) have highlighted the need for explainable, interpretable, and actionable AI-ML. Most work is focused on explaining deep artificial neural networks, e.g., visual and image captioning. In recent work, we established a set of indices and processes for explainable AI (XAI) relative to information fusion. While informative, the result is information overload and domain expertise is required to understand the results. Herein, we explore the extraction of a reduced set of higher-level linguistic summaries to inform and improve communication with non-fusion experts. Our contribution is a proposed structure of a fusion summary and method to extract this information from a given set of indices. In order to demonstrate the usefulness of the proposed methodology, we provide a case study for using the fuzzy integral to combine a heterogeneous set of deep learners in remote sensing for object detection and land cover classification. This case study shows the potential of our approach to inform users about important trends and anomalies in the models, data and fusion results. This information is critical with respect to transparency, trustworthiness, and identifying limitations of fusion techniques, which may motivate future research and innovation.

show abstract

Explainable AI for Understanding Decisions and Data-Driven Optimization of the Choquet Integral

Cited by 13 publications

References 20 publications

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Explainable data fusion

Information Fusion-2-Text: Explainable Aggregation via Linguistic Protoforms

Contact Info

Product

Resources

About