How Does Predicate Invention Affect Human Comprehensibility?

Abstract

This is the accepted version of the paper.This version of the publication may differ from the final published version. Abstract. During the 1980s Michie defined Machine Learning in terms of two orthogonal axes of performance: predictive accuracy and comprehensibility of generated hypotheses. Since predictive accuracy was readily measurable and comprehensibility not so, later definitions in the 1990s, such as that of Mitchell, tended to use a one-dimensional approach to Machine Learning based solely on predictive accuracy, ultimately favouring statistical over symbolic Machine Learning approaches. In this paper we provide a definition of comprehensibility of hypotheses which can be estimated using human participant trials. We present the results of experiments testing human comprehensibility of logic programs learned with and without predicate invention. Results indicate that comprehensibility is affected not only by the complexity of the presented program but also by the existence of anonymous predicate symbols. Permanent repository link

Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP

“…However, to date no documented attempt has been made, even within Inductive Logic Programming, to apply, or demonstrate Michie's ultra-strong criterion to a Machine Learning system. As argued in Schmid et al (2017) the major barrier has been the lack of an operational notion of human comprehension of symbolic concepts.…”

“…In Schmid et al (2017) the issue is addressed by introducing a definition of comprehensibility which is inspired by "Comprehension Tests", administered to school children. Such a test (see Fig.…”

“…In a recent paper (Schmid et al 2017) the authors introduced an operational definition for comprehensibility of logic programs and conducted human trials to determine how properties of a program affect its ease of comprehension. This paper builds on and extends the earlier work by investigating whether machines can not only learn new concepts but explain those concepts to humans thereby improving human task performance.…”

Abstracting probabilistic models: Relations, constraints and beyond

“…Finally, in the context of inductive logic programming and meta-interpretative learning, there is a long history of predicate invention and learning abstract programs [16] and their affect on human comprehensibility [62]. Although the semantic constraints for abstracting is somewhat different, it is possible that our setup regarding partial knowledge in Section 9 could be realized using such methods.…”

“…A parallel concern is about the choosing of a vocabulary that offers maximum comprehensibility [62]. A minor observation to be made here is that if comprehensibility is all we care about (as opposed to being concerned about both comprehensibility and tractability), then it is not clear that one would need to fully abstract a theory.…”