A new algorithm to automate inductive learning of default theories

Abstract

In inductive learning of a broad concept, an algorithm should be able to distinguish concept examples from exceptions and noisy data. An approach through recursively finding patterns in exceptions turns out to correspond to the problem of learning default theories. Default logic is what humans employ in commonsense reasoning. Therefore, learned default theories are better understood by humans. In this paper, we present new algorithms to learn default theories in the form of non-monotonic logic programs. Experiments reported in this paper show that our algorithms are a significant improvement over traditional approaches based on inductive logic programming. Under consideration for acceptance in TPLP. Additionally, ILP is not able to handle exception to general rules: it learns rules under the assumption that there are no exceptions to them. This results in exceptions and noise being treated in the same manner. Often, the exceptions to the rules themselves follow a pattern, and these exceptions can be learned as well. The resulting theory that is learned is a default theory, and in most cases this theory describes the underlying model more accurately. It should be noted that default theories closely model common sense reasoning as well (Baral 2003). Thus, a default theory, if it can be learned, will be more intuitive and comprehensible for humans. Default reasoning also allows us to reason in absence of information. A system that can learn default theories can therefore learn rules that can draw conclusions based on lack of evidence, just like humans. Other reasons that underscore the importance of inductive learning of default theories can be found in Sakama (Sakama 2005) who also surveys other attempts in this direction.As an example, suppose we want to learn the concept of flying ability of birds. We would like to learn the default rule that birds normally fly, as well as rules that capture exceptions, namely, penguins and ostriches are birds that do not fly. Current ILP systems will be thrown off by the exceptions and will not discover any general rule: they will just either enumerate all the birds that fly or cover the positive examples without caring much about the falsely covered negative examples. Other algorithms, such as FOIL, will induce rules that are non-constructive and thus not helpful or intuitive.In this paper, we present two algorithms for learning default theories (i.e., non-monotonic logic programs), called FOLD (First Order Learner of Default) and FOLD-R, to handle categorical and numeric features respectively. Unlike traditional ILP systems that learn standard logic programs (i.e., no negation is allowed), our algorithms learn non-monotonic stratified logic programs (that allow negation-as-failure). Our algorithms are an extension of the FOIL algorithm (Quinlan 1990) and support both categorical and numeric features. Also, the FOLD and FOLD-R learning algorithms can learn recursive rules. Whenever needed, our algorithms introduce new predicates. The language bias (Mitchell 1980) also contains ...

FOLD-R++: A Scalable Toolset for Automated Inductive Learning of Default Theories from Mixed Data

“…The FOLD algorithm [19,20] Example 1 In the FOLD-R++ algorithm, the target is to learn rules for fly(X).…”

“…The FOLD-R algoirthm [19,20] selects the best literal based on the weighted information gain for learning defaults, similar to the original FOLD algorithm described in [20]. For numeric features, the FOLD-R algorithm would enumerate all the possible splits.…”

White-box Induction From SVM Models: Explainable AI with Logic Programming

“…There are number of directions for future work: (i) Equation 7can measure dissimilarity too. Dissimilarity is usually expressed as negation-as-failure in ILP (Shakerin et al 2017). We plan to incorporate dissimilarity to support vectors of the opposite class as a way to reduce the number of induced clauses as well as increase the accuracy of each clause.…”

“…In such a case, we will be learning answer set programs (Gelfond and Kahl 2014). Learning answer set programs has an added advantage of being able to distinguish between exceptions and noise (Shakerin et al 2017;Shakerin and Gupta 2020). (ii) In sparse regions of training data, specialization is usually stopped too early.…”

Induction of Non-Monotonic Rules From Statistical Learning Models Using High-Utility Itemset Mining

“…In this section we present the SHAP-FOLD algorithm. SHAP-FOLD learns a concept in terms of a default theory [15]. A default theory is a non-monotonic logic theory to formalize reasoning with default assumptions in absence of complete information.…”