Towards Interpretable ANNs: An Exact Transformation to Multi-Class Multivariate Decision Trees

Abstract

Deep neural networks (DNNs) are commonly labelled as black-boxes lacking interpretability; thus, hindering human's understanding of DNNs' behaviors. A need exists to generate a meaningful sequential logic for the production of a specific output. Decision trees exhibit better interpretability and expressive power due to their representation language and the existence of efficient algorithms to generate rules. Growing a decision tree based on the available data could produce larger than necessary trees or trees that do not generalise well. In this paper, we introduce two novel multivariate decision tree (MDT) algorithms for rule extraction from a DNN: an Exact-Convertible Decision Tree (EC-DT) and a Deep C-Net algorithm to transform a neural network with Rectified Linear Unit activation functions into a representative tree which can be used to extract multivariate rules for reasoning. While the EC-DT translates the DNN in a layer-wise manner to represent exactly the decision boundaries implicitly learned by the hidden layers of the network, the Deep C-Net inherits the decompositional approach from EC-DT and combines with a C5 tree learning algorithm to construct the decision rules. The results suggest that while EC-DT is superior in preserving the structure and the accuracy of DNN, C-Net generates the most compact and highly effective trees from DNN. Both proposed MDT algorithms generate rules including combinations of multiple attributes for precise interpretation of decision-making processes.

ERIC: Extracting Relations Inferred from Convolutions

“…Early work largely focussed on multi-layer perceptrons (MLPs) with one or very few hidden layers and also on recurrent neural networks. Research has since grown into explaining 'deeper' neural networks of several to many layers, be these MLPs that are deep in this particular sense [26,27,28] or more advanced architectures such as LSTMs [29], Deep Belief Networks [30] or CNNs [16,17,18,19,20,21]. Remaining subsections only cover methods that extract explanations from CNNs.…”