In inductive learning of a broad concept, an algorithm should be able to distinguish concept examples from exceptions and noisy data. An approach through recursively finding patterns in exceptions turns out to correspond to the problem of learning default theories. Default logic is what humans employ in commonsense reasoning. Therefore, learned default theories are better understood by humans. In this paper, we present new algorithms to learn default theories in the form of non-monotonic logic programs. Experiments reported in this paper show that our algorithms are a significant improvement over traditional approaches based on inductive logic programming. Under consideration for acceptance in TPLP. Additionally, ILP is not able to handle exception to general rules: it learns rules under the assumption that there are no exceptions to them. This results in exceptions and noise being treated in the same manner. Often, the exceptions to the rules themselves follow a pattern, and these exceptions can be learned as well. The resulting theory that is learned is a default theory, and in most cases this theory describes the underlying model more accurately. It should be noted that default theories closely model common sense reasoning as well (Baral 2003). Thus, a default theory, if it can be learned, will be more intuitive and comprehensible for humans. Default reasoning also allows us to reason in absence of information. A system that can learn default theories can therefore learn rules that can draw conclusions based on lack of evidence, just like humans. Other reasons that underscore the importance of inductive learning of default theories can be found in Sakama (Sakama 2005) who also surveys other attempts in this direction.As an example, suppose we want to learn the concept of flying ability of birds. We would like to learn the default rule that birds normally fly, as well as rules that capture exceptions, namely, penguins and ostriches are birds that do not fly. Current ILP systems will be thrown off by the exceptions and will not discover any general rule: they will just either enumerate all the birds that fly or cover the positive examples without caring much about the falsely covered negative examples. Other algorithms, such as FOIL, will induce rules that are non-constructive and thus not helpful or intuitive.In this paper, we present two algorithms for learning default theories (i.e., non-monotonic logic programs), called FOLD (First Order Learner of Default) and FOLD-R, to handle categorical and numeric features respectively. Unlike traditional ILP systems that learn standard logic programs (i.e., no negation is allowed), our algorithms learn non-monotonic stratified logic programs (that allow negation-as-failure). Our algorithms are an extension of the FOIL algorithm (Quinlan 1990) and support both categorical and numeric features. Also, the FOLD and FOLD-R learning algorithms can learn recursive rules. Whenever needed, our algorithms introduce new predicates. The language bias (Mitchell 1980) also contains ...

We present a heuristic based algorithm to induce nonmonotonic logic programs that will explain the behavior of XGBoost trained classifiers. We use the technique based on the LIME approach to locally select the most important features contributing to the classification decision. Then, in order to explain the model's global behavior, we propose the LIME-FOLD algorithm -a heuristic-based inductive logic programming (ILP) algorithm capable of learning non-monotonic logic programs-that we apply to a transformed dataset produced by LIME. Our proposed approach is agnostic to the choice of the ILP algorithm. Our experiments with UCI standard benchmarks suggest a significant improvement in terms of classification evaluation metrics. Meanwhile, the number of induced rules dramatically decreases compared to ALEPH, a state-of-the-art ILP system.

No abstract

Answer Set Programming (ASP) has emerged as a successful paradigm for developing intelligent applications. ASP is based on adding negation as failure to logic programming under the stable model semantics regime. ASP allows for sophisticated reasoning mechanisms that are employed by humans to be modeled elegantly. We argue that being able to model common sense reasoning as used by humans is critical for success of automated reasoning. We also argue that extending answer programming systems to general predicates is critical to realizing the full power of ASP. Goal-directed predicate ASP systems are needed to make the ASP technology practical for building large, scalable knowledge-based applications.

We focus on the problem of inducing logic programs that explain models learned by the support vector machine (SVM) algorithm. The top-down sequential covering inductive logic programming (ILP) algorithms (e.g., FOIL) apply hill-climbing search using heuristics from information theory. A major issue with this class of algorithms is getting stuck in local optima. In our new approach, however, the data-dependent hill-climbing search is replaced with a model-dependent search where a globally optimal SVM model is trained first, then the algorithm looks into support vectors as the most influential data points in the model, and induces a clause that would cover the support vector and points that are most similar to that support vector. Instead of defining a fixed hypothesis search space, our algorithm makes use of SHAP, an example-specific interpreter in explainable AI, to determine a relevant set of features. This approach yields an algorithm that captures the SVM model’s underlying logic and outperforms other ILP algorithms in terms of the number of induced clauses and classification evaluation metrics.