An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models

Huysmans, Johan; Dejaeger, Karel; Mues, Christophe; Vanthienen, Jan; Baesens, Bart

doi:10.1016/j.dss.2010.12.003

Cited by 338 publications

(193 citation statements)

References 40 publications

Supporting

Mentioning

174

Contrasting

Unclassified

Order By: Relevance

“…Also, in one of two study cases a negative correlation between the complexity and the understandability of the classification models is found. In a more extensive empirical study, again in the context of data mining, Huysmans et al (2011) investigate the suitability of a number of alternative representation formats for classification when interpretability is a key requirement. In order to compare different standard representation formats, an end-user experiment is designed to test the accuracy, response time, and answer confidence for a set of problem-solving tasks.…”

Section: Empirical Evaluations Of Comprehensibility Of Models and Expmentioning

confidence: 99%

Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP

et al. 2018

View full text Add to dashboard Cite

During the 1980s Michie defined Machine Learning in terms of two orthogonal axes of performance: predictive accuracy and comprehensibility of generated hypotheses. Since predictive accuracy was readily measurable and comprehensibility not so, later definitions in the 1990s, such as Mitchell's, tended to use a one-dimensional approach to Machine Learning based solely on predictive accuracy, ultimately favouring statistical over symbolic Machine Learning approaches. In this paper we provide a definition of comprehensibility of hypotheses which can be estimated using human participant trials. We present two sets of experiments testing human comprehensibility of logic programs. In the first experiment we test human comprehensibility with and without predicate invention. Results indicate comprehensibility is affected not only by the complexity of the presented program but also by the existence of anonymous predicate symbols. In the second experiment we directly test whether any state-of-the-art ILP systems are ultra-strong learners in Michie's sense, and select the Metagol system for use in humans trials. Results show participants were not able to learn Editors: James Cussens and Alessandra Russo. Mach Learn (2018Learn ( ) 107:1119Learn ( -1140 the relational concept on their own from a set of examples but they were able to apply the relational definition provided by the ILP system correctly. This implies the existence of a class of relational concepts which are hard to acquire for humans, though easy to understand given an abstract explanation. We believe improved understanding of this class could have potential relevance to contexts involving human learning, teaching and verbal interaction.

show abstract

Section: Empirical Evaluations Of Comprehensibility Of Models and Expmentioning

confidence: 99%

Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP

et al. 2018

View full text Add to dashboard Cite

show abstract

“…It has been demonstrated that increased pattern length can actually decrease the information gained due to the decreased generality of the patterns , as well as decreasing the interpretability of the patterns (Freitas 2013, Huysmans et al 2011, Letham et al 2013, Vellido et al 2012. Note that the data mining processes used to generate Ψ1 and Ψ2 might have different k's, denoted kΨ 1 and kΨ 2 , but the combinations possible with a smaller k are a subset of a larger k's combinations.…”

Section: Fletcher and Islam 2018 Vol 22 Research Articlementioning

confidence: 99%

Comparing sets of patterns with the Jaccard index

Fletcher

Islam

2018

AJIS

View full text Add to dashboard Cite

The ability to extract knowledge from data has been the driving force of Data Mining since its inception, and of statistical modelling long before even that. Actionable knowledge often takes the form of patterns, where a set of antecedents can be used to infer a consequent. In this paper we offer a solution to the problem of comparing different sets of patterns. Our solution allows comparisons between sets of patterns that were derived from different techniques (such as different classification algorithms), or made from different samples of data (such as temporal data or data perturbed for privacy reasons). We propose using the Jaccard index to measure the similarity between sets of patterns by converting each pattern into a single element within the set. Our measure focuses on providing conceptual simplicity, computational simplicity, interpretability, and wide applicability. The results of this measure are compared to prediction accuracy in the context of a real-world data mining scenario.

show abstract

“…The comprehensibility of the knowledge discovered has been an area of growing interest and it is considered to be as important as obtaining high accuracy [2,28].…”

Section: Rule-based Modelsmentioning

confidence: 99%

Speeding up multiple instance learning classification rules on GPUs

2014

View full text Add to dashboard Cite

Multiple instance learning is a challenging task in supervised learning and data mining. However, algorithm performance becomes slow when learning from large-scale and high-dimensional data sets. Graphics processing units (GPUs) are being used for reducing computing time of algorithms. This paper presents an implementation of the G3P-MI algorithm on GPUs for solving multiple instance problems using classification rules. The GPU model proposed is distributable to multiple GPUs, seeking for its scalability across large-scale and high-dimensional data sets. The proposal is compared to the multi-threaded CPU algorithm with SSE parallelism over a series of data sets. Experimental results report that the computation time can be significantly reduced and its scalability improved. Specifically, an speedup of up to 149× can be achieved over the multi-threaded CPU algorithm when using four GPUs, and the rules interpreter achieves great efficiency and runs over 108 billion Genetic Programming operations per second.

show abstract

An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models

Cited by 338 publications

References 40 publications

Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP

Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP

Comparing sets of patterns with the Jaccard index

Speeding up multiple instance learning classification rules on GPUs

Contact Info

Product

Resources

About