Finding Probabilistic Rule Lists using the Minimum Description Length Principle

Abstract

An important task in data mining is that of rule discovery in supervised data. Well-known examples include rule-based classification and subgroup discovery. Motivated by the need to succinctly describe an entire labeled dataset, rather than accurately classify the label, we propose an MDL-based supervised rule discovery task. The task concerns the discovery of a small rule list where each rule captures the probability of the Boolean target attribute being true. Our approach is built on a novel combination of two main building blocks: (i) the use of the Minimum Description Length (MDL) principle to characterize good-andsmall sets of probabilistic rules, (ii) the use of branch-and-bound with a best-first search strategy to find better-than-greedy and optimal solutions for the proposed task. We experimentally show the effectiveness of our approach, by providing a comparison with other supervised rule learning algorithms on real-life datasets.

The minimum description length principle for pattern mining: a survey

“…They then apply the approach to learn rules about activation patterns in neural networks (Fischer et al 2021). Aoga et al (2018) present a method to encode a binary label associated to each transaction, using the original transactions and a list of rules, each associated to a probability that the target variable holds true. Proença and van Leeuwen (2020a) (also Proença and van Leeuwen 2020b) consider a similar task, but with multiple classes and targeted towards predictive rather than descriptive rules, then looking for rules that capture deviating groups of transactions, i.e.…”

Interpretable multiclass classification by MDL-based rule lists

“…Recent Bayesian methods [44,25,44] are limited to small numbers of candidate rules and binary classification, limiting their usability, and are here represented by SBRL [46] (which is representative for all of them). A recent approach also using MDL and probabilistic rule lists (MRL) [4] is aimed at describing rather than classifying and cannot deal with multiclass problems or a large number of candidates. Interpretable decision sets (IDS) [25] and certifiable optimal rules (CORELS) [3] use similar rules but do not provide probabilistic models or predictions.…”

“…DiffNorm [9] creates models for combinations of classes and also uses the prequential plug-in code, but was designed for data summarization. Aoga et al recently also proposed to use probabilistic rule lists and MDL [4], but 1) we propose a vastly improved encoding, which is tailored towards prediction (instead of summarization), 2) our solution does multiclass classification, and 3) our algorithm has better scalability.…”