Understanding Boolean Function Learnability on Deep Neural Networks

Abstract

Computational learning theory states that many classes of boolean formulas are learnable in polynomial time. This paper addresses the understudied subject of how, in practice, such formulas can be learned by deep neural networks. Specifically, we analyse boolean formulas associated with the decision version of combinatorial optimisation problems, model sampling benchmarks, and random 3-CNFs with varying degrees of constrainedness. Our extensive experiments indicate that: (i) regardless of the combinatorial optimisation problem, relatively small and shallow neural networks are very good approximators of the associated formulas; (ii) smaller formulas seem harder to learn, possibly due to the fewer positive (satisfying) examples available; and (iii) interestingly, underconstrained 3-CNF formulas are more challenging to learn than overconstrained ones. Source code and relevant datasets are publicly available 1 .

Logic Explained Networks

“…Experiments of Table 4 show how all the considered out-of-the-box LENs generalize better than decision trees on complex Boolean functions (e.g. CUB) as expected [Tavares et al, 2020], and they usually outperform BRL as well. LENs based on ReLU networks are definitely the ones with better classification accuracy, confirming the intuitions reported in Section 5.3. µ nets, however, are quite close in terms of classification accuracy.…”

“…• It reports experimental results using three out-of-the-box preset LENs showing how they may generalize better in terms of model accuracy than established white-box models such as decision trees on complex Boolean tasks (in line with Tavares' work [Tavares et al, 2020]).…”

“…In principle, interpretable models are the best suited to be employed as decision support systems. However, their decision function often is not smooth which makes them quite sensible to data distribution shifts and impairs their generalization ability on unseen observations [Tavares et al, 2020, Molnar, 2020. On the other hand, explanation methods can be applied to get approximated interpretations of state-of-the-art models.…”

Entropy-Based Logic Explanations of Neural Networks

“…classification accuracy, explanation accuracy, and fidelity) can be explained observing how entropy-based networks are far less constrained than ψ networks, both in the architecture (our approach does not apply weight pruning) and in the loss function (our approach applies a regularization on the distributions α i and not on all weight matrices). Likewise, the main reason why the proposed approach provides a higher classification accuracy with respect to BRL and decision trees may lie in the smoothness of the decision functions of neural networks which tend to generalize better than rule-based methods, as already observed by Tavares et al (Tavares et al 2020) report in the Appendix a few examples of logic explanations extracted by each method, as well as in Fig. 4.…”