DeepNNK: Explaining deep models and their generalization using polytope interpolation

Abstract

Modern machine learning systems based on neural networks have shown great success in learning complex data patterns while being able to make good predictions on unseen data points. However, the limited interpretability of these systems hinders further progress and application to several domains in the real world. This predicament is exemplified by time consuming model selection and the difficulties faced in predictive explainability, especially in the presence of adversarial examples. In this paper, we take a step towards better understanding of neural networks by introducing a local polytope interpolation method. The proposed Deep Non Negative Kernel regression (NNK) interpolation framework is non parametric, theoretically simple and geometrically intuitive. We demonstrate instance based explainability for deep learning models and develop a method to identify models with good generalization properties using leave one out estimation. Finally, we draw a rationalization to adversarial and generative examples which are inevitable from an interpolation view of machine learning.

Channel-Wise Early Stopping without a Validation Set via NNK Polytope Interpolation

“…DeepNNK [7] is a non-parametric interpolation framework based on local polytopes obtained using NNK graphs [10] that replaces the standard softmax classification layer of a neural network using the activations obtained at the penultimate layer as input features, for model evaluation and inference. A key advantage of DeepNNK lies in the fact that label interpolation is performed based on the relative positions of points in the training set, which makes it possible to perform leave one out estimation to characterize the task performance of a model and its generalization without the need for additional data, namely the validation set.…”

“…Training is stopped for a given channel if its corresponding LOO classification performance no longer improves. Our proposed method achieves comparable test performance with fewer training iterations than existing methods, such as validation-set-based early stopping and aggregate feature vector based DeepNNK [7]. Our early stopping criterion requires lower runtimes and has the further advantage of not requiring any validation data to be set aside.…”

“…The ability of graphs to define relationships between different types of entities allows us to describe and analyze complex patterns in data [3]. Recently, graphs have been used to understand and improve intermediate representations of deep learning models with application to various tasks, such as model regularization [4] and robustness [5], model distillation [6], and model interpretation [7], [8]. Since no graph is given a priori, these methods typically begin with a graph construction phase, where each graph node corresponds to an item in the training set, and the weight of an edge between two nodes is a function of the distance between their respective intermediate layer activations (i.e., their respective feature vectors).…”

“…In particular, we propose a channel-wise extension of DeepNNK [7], CW-DeepNNK, which leverages the geometrical interpretation and the robustness of NNK [10] to obtain per channel leave one out (LOO) classification estimates. Our analysis allows us to show that each channel learns specific features, each having different levels of importance for the overall task.…”

Channel redundancy and overlap in convolutional neural networks with channel-wise NNK graphs

“…Further, NNK neighborhoods are stable for large enough K, and their size is indicative of the ID of the manifold the data belongs to [18]. This method has been shown to provide advantages for semi-supervised learning, image representation [19], and label interpolation and generalization estimation in neural networks [20]. Graph properties have also been used to interpret deep neural network performance [21], latent space geometry [22,23], and to improve model robustness [24].…”