On the quest for optimal rule learning heuristics

Abstract

The primary goal of the research reported in this paper is to identify what criteria are responsible for the good performance of a heuristic rule evaluation function in a greedy top-down covering algorithm. We first argue that search heuristics for inductive rule learning algorithms typically trade off consistency and coverage, and we investigate this trade-off by determining optimal parameter settings for five different parametrized heuristics. In order to avoid biasing our study by known functional families, we also investigate the potential of using metalearning for obtaining alternative rule learning heuristics. The key results of this experimental study are not only practical default values for commonly used heuristics and a broad comparative evaluation of known and novel rule learning heuristics, but we also gain theoretical insights into factors that are responsible for a good performance. For example, we observe that consistency should be weighted more heavily than coverage, presumably because a lack of coverage can later be corrected by learning additional rules.

A study of different quality evaluation functions in the cAnt-Miner(PB) classification algorithm

“…The importance of the rule quality function in sequential covering algorithms has been highlighted in [4,6], and in the context of ACO classification algorithm in [12]. In this paper we present a study of the effects of rule quality functions, as well as list quality functions, in the cAnt-MinerPB algorithm.…”

“…Sequential covering, also called separate-and-conquer, is a classification rule learning approach with two main discrete steps [6]. In essence, the approach finds a rule with a high quality on the dataset (conquer), and then removes the examples which are covered by the rule (separate).…”

“…The effectiveness of a sequential covering algorithm stems from the quality of the rules that are produced [4,6]. Some rules may only cover correctly classified examples (high consistency) but might only cover a small sample (low coverage).…”

“…It has been suggested that traditional non-parametric rule quality functions, such as Sensitivity × Specificity, accuracy or precision, implement a fixed trade off [6]. That is, they either prefer consistency or coverage, with no means of modifying this bias.…”

“…One is a mainly theoretical study [6], which looks at how rule quality functions affect a relatively simple sequential search algorithm. In this study, it was shown that the parametrized functions performed well.…”