An Empirical Investigation of the Trade-Off between Consistency and Coverage in Rule Learning Heuristics

Janssen, Frederik; Fürnkranz, Johannes

doi:10.1007/978-3-540-88411-8_7

Cited by 6 publications

(11 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Precision, for example, tends to learn many rules which usually contain a lot of conditions. On the contrary, Weighted Relative Accuracy (WRA) [15] often settles for very few rules that are overly general [22,13]. Correlation, as another example, computes the correlation coefficient between the predicted and the target labels.…”

Section: Rule Learning Heuristicsmentioning

confidence: 99%

“…Two heuristics, m-estimate [3] and the relative costs measure [9], have parameters which may be viewed as a means for trading off between optimizing consistency and coverage. In a recent study [13], we have optimized the parameters of these heuristics for hillclimbing search. Based on these results, we will use the settings m = 22.466 for the m-estimate, and c = 0.342 for the cost measure.…”

Section: Rule Learning Heuristicsmentioning

confidence: 99%

“…The key difference to previous work is that we evaluate nine different heuristics which have been recently evaluated for heuristic search [13]. Most of them are well-known in this context, but have not been used for exhaustive search before.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Re-evaluation of the Over-Searching Phenomenon in Inductive Rule Learning

Janssen

Fürnkranz

2009

Proceedings of the 2009 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

Most commonly used inductive rule learning algorithms employ a hill-climbing search, whereas local pattern discovery algorithms employ exhaustive search. In this paper, we evaluate the spectrum of different search strategies to see whether separate-and-conquer rule learning algorithms are able to gain performance in terms of predictive accuracy or theory size by using more powerful search strategies like beam search or exhaustive search. Unlike previous results that demonstrated that rule learning algorithms suffer from over-searching, our work pays particular attention to the interaction between the search heuristic and the search strategy. Our results show that exhaustive search has primarily the effect of finding longer, but nevertheless more general rules than hill-climbing search. Thus, in cases where hillclimbing finds too specific rules, exhaustive search may help, while in others it may lead to over-generalization.

show abstract

Section: Rule Learning Heuristicsmentioning

confidence: 99%

Section: Rule Learning Heuristicsmentioning

confidence: 99%

See 1 more Smart Citation

A Re-evaluation of the Over-Searching Phenomenon in Inductive Rule Learning

Janssen

Fürnkranz

2009

Proceedings of the 2009 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

show abstract

“…Parts of this paper have previously appeared in Janssen and Fürnkranz (2008) and Janssen and Fürnkranz (2007).…”

Section: Introductionmentioning

confidence: 99%

On the quest for optimal rule learning heuristics

Janssen

Fürnkranz

2009

Mach Learn

Self Cite

View full text Add to dashboard Cite

The primary goal of the research reported in this paper is to identify what criteria are responsible for the good performance of a heuristic rule evaluation function in a greedy top-down covering algorithm. We first argue that search heuristics for inductive rule learning algorithms typically trade off consistency and coverage, and we investigate this trade-off by determining optimal parameter settings for five different parametrized heuristics. In order to avoid biasing our study by known functional families, we also investigate the potential of using metalearning for obtaining alternative rule learning heuristics. The key results of this experimental study are not only practical default values for commonly used heuristics and a broad comparative evaluation of known and novel rule learning heuristics, but we also gain theoretical insights into factors that are responsible for a good performance. For example, we observe that consistency should be weighted more heavily than coverage, presumably because a lack of coverage can later be corrected by learning additional rules.

show abstract

“…Similar to our experiments, empirical studies aimed at discovering optimal rule learning heuristics have been published in the realm of single-label classification [14,15]. Moreover, to investigate the properties of bipartition evaluation functions, ROC space isometrics have been proven to be a helpful tool [9,11].…”

Section: Related Workmentioning

confidence: 95%

On the Trade-Off Between Consistency and Coverage in Multi-label Rule Learning Heuristics

2019

Self Cite

View full text Add to dashboard Cite

Recently, several authors have advocated the use of rule learning algorithms to model multi-label data, as rules are interpretable and can be comprehended, analyzed, or qualitatively evaluated by domain experts. Many rule learning algorithms employ a heuristic-guided search for rules that model regularities contained in the training data and it is commonly accepted that the choice of the heuristic has a significant impact on the predictive performance of the learner. Whereas the properties of rule learning heuristics have been studied in the realm of single-label classification, there is no such work taking into account the particularities of multi-label classification. This is surprising, as the quality of multi-label predictions is usually assessed in terms of a variety of different, potentially competing, performance measures that cannot all be optimized by a single learner at the same time. In this work, we show empirically that it is crucial to trade off the consistency and coverage of rules differently, depending on which multi-label measure should be optimized by a model. Based on these findings, we emphasize the need for configurable learners that can flexibly use different heuristics. As our experiments reveal, the choice of the heuristic is not straight-forward, because a search for rules that optimize a measure locally does usually not result in a model that maximizes that measure globally.

show abstract

An Empirical Investigation of the Trade-Off between Consistency and Coverage in Rule Learning Heuristics

Cited by 6 publications

References 8 publications

A Re-evaluation of the Over-Searching Phenomenon in Inductive Rule Learning

A Re-evaluation of the Over-Searching Phenomenon in Inductive Rule Learning

On the quest for optimal rule learning heuristics

On the Trade-Off Between Consistency and Coverage in Multi-label Rule Learning Heuristics

Contact Info

Product

Resources

About