An Efficient Iterative Approach to Explainable Feature Learning

Abstract

This article introduces a new iterative approach to explainable feature learning. During each iteration, new features are generated, first by applying arithmetic operations on the input set of features. These are then evaluated in terms of probability distribution agreements between values of samples belonging to different classes. Finally, a graph-based approach for feature selection is proposed, which allows for selecting high-quality and uncorrelated features to be used in feature generation during the next iteration. As shown by the results, the proposed method improved the accuracy of all tested classifiers, where the best accuracies were achieved using random forest. In addition, the method turned out to be insensitive to both of the input parameters, while superior performances in comparison to the state of the art were demonstrated on nine out of 15 test sets and achieving comparable results in the others. Finally, we demonstrate the explainability of the learned feature representation for knowledge discovery.

A New Alternating Suboptimal Dynamic Programming Algorithm with Applications for Feature Selection

“…Our method processes, e.g., 200 features in 5 seconds, but for larger input sets, it makes sense to preprocess it with some faster filtering. We use our efficient and reliable graph cut-based feature selection [58], summarised in Subsection 3.1. In Subsection 3.2, we discuss the idea of using DP and the encountered difficulties and introduce an iterative suboptimal alternating solution, where the order of feature processing is inverted in each iteration.…”

“…This section describes a graph cut-based feature selection approach, presented in [58] that allows for extracting a subset of high-quality dissimilar features. Depending on the defined feature estimation measurement, it can be used for classification and regression purposes.…”

“…Depending on the defined feature estimation measurement, it can be used for classification and regression purposes. Graph vertices represent features with associated weights that define their quality (as proposed in [58]), while graph edge weights define similarities between them. The method relies on two input parameters, T ∆ and T p , used for graph definition.…”

“…An undirected graph used for feature selection is defined as G = (F, E), where a set of vertices F is defined as F = { f i ∈ FS; ∆( f i ) ≤ T ∆ }, while an unordered set of edges E = e i,j ; P(e i,j ) ≥ T p is given by e i,j = ( f i , f j ) for all f i , f j ∈ F, such that i ̸ = j. A vertex-weighting function is given by ∆( f i ), as defined in [58], and the edge-weighting function is given by the absolute Pearson correlation coefficient P : E → [0, 1], formally described by (1).…”

Cluster-CAM: Cluster-Weighted Visual Interpretation of CNNs' Decision in Image Classification

“…Despite CNNs' extraordinary performance, they still lack a clear interpretation of the inner mechanism [10,42,21]. This lack of transparency can indeed be a disqualifying factor in some peculiar scenarios where mistakes in interpretation can jeopardize human life and health, like in medical image processing or autonomous vehicles [20,38,34,32]. Therefore, it is highly desirable to find a way to understand and explain what exactly CNNs have learned during the training process [26,31,16].…”