No abstract

While neural networks have been achieving increasingly significant excitement in solving classification tasks such as natural language processing, their lack of interpretability becomes a great challenge for neural networks to be deployed in certain high-stakes human-centered applications. To address this issue, we propose a new approach for generating interpretable predictions by inferring a simple three-layer neural network with threshold activations, so that it can benefit from effective neural network training algorithms and at the same time, produce human-understandable explanations for the results. In particular, the hidden layer neurons in the proposed model are trained with floating point weights and binary output activations. The output neuron is also trainable as a threshold logic function that implements a disjunctive operation, forming the logical-OR of the first-level threshold logic functions. This neural network can be trained using state-of-the-art training methods to achieve high prediction accuracy. An important feature of the proposed architecture is that only a simple greedy algorithm is required to provide an explanation with the prediction that is human-understandable. In comparison with other explainable decision models, our proposed approach achieves more accurate predictions on a broad set of tabular data classification datasets.

Tabular datasets can be viewed as logic functions that can be simplified using two-level logic minimization to produce minimal logic formulas in disjunctive normal form (DNF), which in turn can be readily viewed as an explainable decision rule set for binary classification. However, there are two problems with using logic minimization for tabular machine learning. First, tabular datasets often contain overlapping examples that have different class labels, which have to be resolved before logic minimization can be applied since logic minimization assumes consistent logic functions. Second, even without inconsistencies, logic minimization alone generally produces complex models with poor generalization because it exactly fits all data points, which leads to detrimental overfitting. How best to remove training instances to eliminate inconsistencies and overfitting is highly nontrivial. In this paper, we propose a novel statistical framework for removing these training samples so that logic minimization can become an effective approach to tabular machine learning. Using the proposed approach, we are able to obtain comparable performance as gradient boosted and ensemble decision trees, which have been the winning hypothesis classes in tabular learning competitions, but with human-understandable explanations in the form of decision rules. To our knowledge, neither logic minimization nor explainable decision rule methods have been able to achieve state-of-the-art performance before in tabular learning problems.Impact Statement-Decision rule sets are an important hypothesis class for tabular learning problems in which the ability to provide human understandable explanations is of critical importance. However, they are generally not the winning hypothesis class in terms of accuracy. Black-box models like gradient boosted and ensemble decision trees are generally the superior models. In this paper, we revisit the use of logic minimization to derive explainable decision rule sets from tabular datasets. Logic minimization alone produces complex models with poor generalization because it exactly fits all data points as provided. We overcome this problem by removing instances that cause inconsistencies and overfitting via a novel statistical framework. The proposed approach makes possible the learning of decision rules that achieve state-of-the-art classification performance in tabular learning problems with explainable rule-based predictions, which has not been achieved before.

Conventional federated learning (FL) approaches are ineffective in scenarios where clients have significant differences in the distributions of their local data. The Non-IID data distribution in the client data causes a drift in the local model updates from the global optima, which significantly impacts the performance of the trained models. In this paper, we present a new algorithm called FLIS that aims to address this problem by grouping clients into clusters that have jointly trainable data distributions. This is achieved by comparing the inference similarity of client models. Our proposed framework captures settings where different groups of users may have their own objectives (learning tasks), but by aggregating their data with others in the same cluster (same learning task), superior models can be derived via more efficient and personalized federated learning. We present experimental results to demonstrate the benefits of FLIS over the state-of-the-art approaches on the CIFAR-100/10, SVHN, and FMNIST datasets. Our code is available at https://github.com/MMorafah/FLIS.