Logic Explained Networks

Abstract

The large and still increasing popularity of deep learning clashes with a major limit of neural network architectures, that consists in their lack of capability in providing human-understandable motivations of their decisions. In situations in which the machine is expected to support the decision of human experts, providing a comprehensible explanation is a feature of crucial importance. The language used to communicate the explanations must be formal enough to be implementable in a machine and friendly enough to be understandable by a wide audience. In this paper, we propose a general approach to Explainable Artificial Intelligence in the case of neural architectures, showing how a mindful design of the networks leads to a family of interpretable deep learning models called Logic Explained Networks (LENs). LENs only require their inputs to be human-understandable predicates, and they provide explanations in terms of simple First-Order Logic (FOL) formulas involving such predicates. LENs are general enough to cover a large number of scenarios. Amongst them, we consider the case in which LENs are directly used as special classifiers with the capability of being explainable, or when they act as additional networks with the role of creating the conditions for making a black-box classifier explainable by FOL formulas. Despite supervised learning problems are mostly emphasized, we also show that LENs can learn and provide explanations in unsupervised learning settings. Experimental results on several datasets and tasks show that LENs may yield better classifications than established white-box models, such as decision trees and Bayesian rule lists, while providing more compact and meaningful explanations.

Knowledge-driven Active Learning

“…The proposed approach could be refined by asking for supervision only for the predicates involved in the violated rules, reducing further the number of required labelled data and leading to a more balanced training set improving the prediction on smaller classes. At last, in case no knowledge is available on a certain problem, a first idea could be to pair the KAL strategy with the method proposed in Ciravegna et al (2021), where FOL explanations of network predictions are extracted on training data, to continuously check whether the knowledge learnt on the training distribution is also valid on unseen data.…”

Encoding Concepts in Graph Neural Networks

“…Predicting tasks as a function of learnt concepts makes the decision process of deep learning models more interpretable [20,1]. In fact, learning intermediate concepts allows models to provide concept-based explanations for their predictions [17] which can take the form of simple logic statements, as shown by Ciravegna et al [26]. In addition, Koh et al [20] showed how learning intermediate concepts allows human experts to rectify mispredicted concepts through effective test-time interventions, thus improving model's performance and engendering human trust [1].…”

“…The second CEM step consists of using the extracted concepts to make interpretable predictions for downstream tasks. In particular, the presence of concepts enables pairing GNNs with existing concept-based methods which are explainable by design, such as Logic Explained Networks (LENs, [26]). LENs are neural models providing simple concept-based logic explanations for their predictions.…”

Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations

“…Several studies highlight the benefits of concept-based machine learning for explainability [1,82,23,52] and human interactions [75]. To communicate the concepts to a human user, some approaches include first-order logic formulas [16], causal relationships [81], user defined concepts [40], prediction of intermediate dataset-labels [3,52], and one-hot encoded bottlenecks [42]. All of these approaches, however, focus on supervised concept learning.…”