A Survey of the State of Explainable AI for Natural Language Processing

Danilevsky, Marina; Qian, Kun; Aharonov, Ranit; Katsis, Yannis; Kawas, Ban; Sen, Prithviraj

doi:10.48550/arxiv.2010.00711

Cited by 36 publications

(44 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Explainability is important since we want to be able to trust that our agents are beneficial before deploying them. For a recent survey of explainable natural language processing (NLP), see Danilevsky et al (2020). Note that explainability doesn't come for free -there still needs to be incentives for language agents to give true and useful explanations of their behaviour.…”

Section: Language Agentsmentioning

confidence: 99%

Alignment of Language Agents

Kenton,

Everitt,

Weidinger

et al. 2021

Preprint

View full text Add to dashboard Cite

For artificial intelligence to be beneficial to humans the behaviour of AI agents needs to be aligned with what humans want. In this paper we discuss some behavioural issues for language agents, arising from accidental misspecification by the system designer. We highlight some ways that misspecification can occur and discuss some behavioural issues that could arise from misspecification, including deceptive or manipulative language, and review some approaches for avoiding these issues.

show abstract

Section: Language Agentsmentioning

confidence: 99%

Alignment of Language Agents

Kenton,

Everitt,

Weidinger

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A local explanation justifies a model or method's output for a specific input. A global explanation provides a justification on average performance of a model, independently of any particular input [20]. There are techniques for explainability using visualization of neuronal activity in layers of a deep architecture, for examples, based on sparsity and heatmaps [21], [22], [23].…”

Section: A Relevant Literaturementioning

confidence: 99%

“…We refer to the forward mapping as the transformation from the data space to the latent space, and the reverse mapping as the transformation from the latent space to the data space. The log-determinant is usually computed for the forward mapping at each of the constituent layers shown in (20). Each of the individual layers of a single flow-step are described as follows: a) Activation normalization: It is assumed that an input vector is represented as x and is of the shape…”

Section: Generative 1 × 1 Convolution Flow (Glow)mentioning

confidence: 99%

“…For Glow-HMM, we define a similar concept in terms of the parameters K and L, which refers to the number of flow-steps and number of layers of multi-scale flow respectively. One flow-step in the Glow model consists of three layers: activation normalization, invertible convolution, and an affine coupling layer (similar to RealNVP flow) shown in (20). We avoided the use of multi-scale flows and chose the value of K through experiments.…”

Section: Training Of Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability

Ghosh,

Honoré,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

In pursuit of explainability, we develop generative models for sequential data. The proposed models provide state-of-the-art classification results and robust performance for speech phone classification. We combine modern neural networks (normalizing flows) and traditional generative models (hidden Markov models -HMMs). Normalizing flow-based mixture models (NMMs) are used to model the conditional probability distribution given the hidden state in the HMMs. Model parameters are learned through judicious combinations of time-tested Bayesian learning methods and contemporary neural network learning methods. We mainly combine expectation-maximization (EM) and mini-batch gradient descent. The proposed generative models can compute likelihood of a data and hence directly suitable for maximum-likelihood (ML) classification approach. Due to structural flexibility of HMMs, we can use different normalizing flow models. This leads to different types of HMMs providing diversity in data modeling capacity. The diversity provides an opportunity for easy decision fusion from different models. For a standard speech phone classification setup involving 39 phones (classes) and the TIMIT dataset, we show that the use of standard features called mel-frequency-cepstral-coeffcients (MFCCs), the proposed generative models, and the decision fusion together can achieve 86.6% accuracy by generative training only. This result is close to state-of-the-art results, for examples, 86.2% accuracy of PyTorch-Kaldi toolkit [1], and 85.1% accuracy using light gated recurrent units [2]. We do not use any discriminative learning approach and related sophisticated features in this article.

show abstract

“…In this context, a flourishing literature proposing interpretability methods emerged. We refer to the survey papers of Guidotti et al (2018) and Adadi and Berrada (2018) for an overview, and to Danilevsky et al (2020) for a focus on natural language processing. With the notable exception of SHAP (Lundberg and Lee, 2017), these methods do not come with any guarantees.…”

Section: Introductionmentioning

confidence: 99%

An Analysis of LIME for Text Data

Mardaoui,

Garreau

2020

Preprint

View full text Add to dashboard Cite

Text data are increasingly handled in an automated fashion by machine learning algorithms. But the models handling these data are not always well-understood due to their complexity and are more and more often referred to as "black-boxes." Interpretability methods aim to explain how these models operate. Among them, LIME has become one of the most popular in recent years. However, it comes without theoretical guarantees: even for simple models, we are not sure that LIME behaves accurately. In this paper, we provide a first theoretical analysis of LIME for text data. As a consequence of our theoretical findings, we show that LIME indeed provides meaningful explanations for simple models, namely decision trees and linear models.

show abstract

A Survey of the State of Explainable AI for Natural Language Processing

Cited by 36 publications

References 66 publications

Alignment of Language Agents

Alignment of Language Agents

Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability

An Analysis of LIME for Text Data

Contact Info

Product

Resources

About