A modularized framework for explaining hierarchical attention networks on text classifiers

Abstract

The last decade has witnessed the rise of a black box society where classification models that hide the logic of their internal decision processes are widely adopted due to their high accuracy. In this paper, we propose FEHAN, a modularized Framework for Explaining HiErarchical Attention Network trained to classify text data. Given a document, FEHAN extracts sentences most relevant to the assigned class. It then generates a set of similar sentences using a Markov chain text generator, and it replaces the salient sentences with the synthetic ones, resulting in a new set of semantically similar documents in the vicinity of a given instance. The generated documents are used to train an interpretable decision tree that identifies words explaining the reason for the classification outcome. A quick inspection of these synthetic documents and their salient words helps explain why the black-box has assigned a given class to a document. We performed a qualitative and quantitative evaluation of FEHAN and a baseline on four different datasets to show the effectiveness of our proposal.

A Modularized Framework for Explaining Black Box Classifiers for Text Data

“…, w m ⟩, with 1 ≤ i ≤ n and where any w j with 1 ≤ j ≤ m) is a word. Explaining the decision of a black box model f on a given document d, i.e., f (d) = y, means presenting an explanation e, that belongs to a human-understandable domain E. The proposed explanation method is the next step in the line of research on local model-agnostic methods originated from [9,12,21]. Thus, the idea of DICTA is to unveil the reason for classification of a trained text classifier by studying its behavior on the synthetic neighborhood of a given document.…”