Using Rule Extraction to Improve the Comprehensibility of Predictive Models

Huysmans, Johan; Baesens, Bart; Vanthienen, Jan

doi:10.2139/ssrn.961358

Cited by 64 publications

(45 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In Huysmans et al (2006), the authors provided a wide range of survey on different rule extraction methods. Most of those rule extraction methods were designed for some "black box" type classifiers such as Artificial Neural Networks and Support Vector Machines (Martens et al (2008), Schmitz et al (1999)).…”

Section: Australasian Journal Of Information Systems Adnan and Islam 20mentioning

confidence: 99%

“…Most of those rule extraction methods were designed for some "black box" type classifiers such as Artificial Neural Networks and Support Vector Machines (Martens et al (2008), Schmitz et al (1999)). On the contrary, rule extraction from decision forests remains largely ignored (Huysmans et al (2006), Mashayekhi and Gras (2015)). Some existing rule extraction techniques (Liu et al (2012), Mashayekhi and Gras (2015)) mainly prune forest rules in order to increase the prediction accuracy (just like pruning trees from a forest) and consequently do not solely focus on issues related to knowledge discovery.…”

Section: Australasian Journal Of Information Systems Adnan and Islam 20mentioning

confidence: 99%

“…Hence, both Acc and Cov need to be considered when extracting a subset of rules. However, in literature we find no method considering both Acc and Cov for extracting subsets of rules from decision forests (Huysmans et al (2006)). The recently proposed RF+HC method (Mashayekhi and Gras (2015)) considers the number of correctly and incorrectly classified records that are covered by a rule (in effect, accuracy) and the rule length for calculating a score for the rule (see Equation 1).…”

Section: Basic Conceptsmentioning

confidence: 99%

See 2 more Smart Citations

ForEx++: A New Framework for Knowledge Discovery from Decision Forests

Adnan

Islam

2017

AJIS

View full text Add to dashboard Cite

Decision trees are popularly used in a wide range of real world problems for both prediction and classification (logic) rules discovery. A decision forest is an ensemble of decision trees and it is often built for achieving better predictive performance compared to a single decision tree. Besides improving predictive performance, a decision forest can be seen as a pool of logic rules (rules) with great potential for knowledge discovery. However, a standard-sized decision forest usually generates a large number of rules that a user may not able to manage for effective knowledge analysis. In this paper, we propose a new, data set independent framework for extracting those rules that are comparatively more accurate, generalized and concise than others. We apply the proposed framework on rules generated by two different decision forest algorithms from some publicly available medical related data sets on dementia and heart disease. We then compare the quality of rules extracted by the proposed framework with rules generated from a single J48 decision tree and rules extracted by another recent method. The results reported in this paper demonstrate the effectiveness of the proposed framework.

show abstract

Section: Australasian Journal Of Information Systems Adnan and Islam 20mentioning

confidence: 99%

Section: Australasian Journal Of Information Systems Adnan and Islam 20mentioning

confidence: 99%

Section: Basic Conceptsmentioning

confidence: 99%

See 1 more Smart Citation

ForEx++: A New Framework for Knowledge Discovery from Decision Forests

Adnan

Islam

2017

AJIS

View full text Add to dashboard Cite

show abstract

“…To allow for rules of different length, a fairly technical encoding of the rules is used. One other cited drawback of GEX [14] is that one instance can be covered by several rules, while it is not guaranteed that at least one rule will be valid.…”

Section: Related Workmentioning

confidence: 99%

Genetic rule extraction optimizing brier score

Johansson

König

Niklasson

2010

Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

Most highly accurate predictive modeling techniques produce opaque models. When comprehensible models are required, rule extraction is sometimes used to generate a transparent model, based on the opaque. Naturally, the extracted model should be as similar as possible to the opaque. This criterion, called fidelity, is therefore a key part of the optimization function in most rule extracting algorithms. To the best of our knowledge, all existing rule extraction algorithms targeting fidelity use 0/1 fidelity, i.e., maximize the number of identical classifications. In this paper, we suggest and evaluate a rule extraction algorithm utilizing a more informed fidelity criterion. More specifically, the novel algorithm, which is based on genetic programming, minimizes the difference in probability estimates between the extracted and the opaque models, by using the generalized Brier score as fitness function. Experimental results from 26 UCI data sets show that the suggested algorithm obtained considerably higher accuracy and significantly better AUC than both the exact same rule extraction algorithm maximizing 0/1 fidelity, and the standard tree inducer J48. Somewhat surprisingly, rule extraction using the more informed fidelity metric normally resulted in less complex models, making sure that the improved predictive performance was not achieved on the expense of comprehensibility.

show abstract

“…The potential size of a rule for a unit with n inputs each having k possible values is k n , meaning that a straightforward search for possible rules is normally impossible for larger networks. Consequently, most modern rule extraction algorithms are black-box, see the more recent survey [10].…”

Section: Introductionmentioning

confidence: 99%

Rule Extraction with Guaranteed Fidelity

Johansson

König

Linusson

et al. 2014

Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications

View full text Add to dashboard Cite

Part 3: COPA WorkshopInternational audienceThis paper extends the conformal prediction framework to rule extraction, making it possible to extract interpretable models from opaque models in a setting where either the infidelity or the error rate is bounded by a predefined significance level. Experimental results on 27 publicly available data sets show that all three setups evaluated produced valid and rather efficient conformal predictors. The implication is that augmenting rule extraction with conformal prediction allows extraction of models where test set errors or test sets infidelities are guaranteed to be lower than a chosen acceptable level. Clearly this is beneficial for both typical rule extraction scenarios, i.e., either when the purpose is to explain an existing opaque model, or when it is to build a predictive model that must be interpretable

show abstract

Using Rule Extraction to Improve the Comprehensibility of Predictive Models

Cited by 64 publications

References 28 publications

ForEx++: A New Framework for Knowledge Discovery from Decision Forests

ForEx++: A New Framework for Knowledge Discovery from Decision Forests

Genetic rule extraction optimizing brier score

Rule Extraction with Guaranteed Fidelity

Contact Info

Product

Resources

About