Analyzing Data Complexity Using Metafeatures for Classification Algorithm Selection

Shah, Rushit; Khemani, Varun; Azarian, Michael H.; Pecht, Michael; Su, Yan

doi:10.1109/phm-chongqing.2018.00224

Cited by 5 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, other works focused on analyzing and creating frameworks for meta-features, without diving deep into the algorithm selection problem [59][60][61][62][63].…”

Section: Clarifications On the Excluded Recordsmentioning

confidence: 99%

Explainable Rules and Heuristics in AI Algorithm Recommendation Approaches—A Systematic Literature Review and Mapping Study

Jos�Garc韆-Pe馻lvo¹,

V醶quez-Ingelmo²,

Garc韆-Holgado³

2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

The exponential use of artificial intelligence (AI) to solve and automated complex tasks has catapulted its popularity generating some challenges that need to be addressed. While AI is a powerful means to discover interesting patterns and obtain predictive models, the use of these algorithms comes with a great responsibility, as an incomplete or unbalanced set of training data or an unproper interpretation of the models' outcomes could result in misleading conclusions that ultimately could become very dangerous. For these reasons, it is important to rely on expert knowledge when applying these methods. However, not every user can count on this specific expertise; non-AIexpert users could also benefit from applying these powerful algorithms to their domain problems, but they need basic guidelines to obtain the most out of AI models. The goal of this work is to present a systematic review of the literature to analyze studies whose outcomes are explainable rules and heuristics to select suitable AI algorithms given a set of input features. The systematic review follows the methodology proposed by Kitchenham and other authors in the field of software engineering. As a result, 9 papers that tackle AI algorithm recommendation through tangible and traceable rules and heuristics were collected. The reduced number of retrieved papers suggests a lack of reporting explicit rules and heuristics when testing the suitability and performance of AI algorithms.

show abstract

“…On the other hand, other works focused on analyzing and creating frameworks for meta-features, without diving deep into the algorithm selection problem [59][60][61][62][63].…”

Section: Clarifications On the Excluded Recordsmentioning

confidence: 99%

Explainable Rules and Heuristics in AI Algorithm Recommendation Approaches—A Systematic Literature Review and Mapping Study

Jos�Garc韆-Pe馻lvo¹,

V醶quez-Ingelmo²,

Garc韆-Holgado³

2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

show abstract

“…More recent works on meta-learning include: Cruz et al[2015], dasDôres et al [2016],Roy et al [2016],Parmezan et al [2017], Cruz et al [2017a,Garcia et al [2018],Shah et al [2018]. In[Garcia et al, 2018], for example, all of the complexity measures described in this work were employed to generate regression models able to predict the accuracies of four classifiers with very distinct biases: ANN, decision tree, kNN and SVM.…”

mentioning

confidence: 99%

“…Mollineda et al [2006],Walt and Barnard [2007], Krijthe et al [2012], Ren and Vale [2012] das Dôres et al [2016], Roy et al [2016], Parmezan et al [2017], Cruz et al [2015, 2017a] Muñoz et al [2018], Zhang et al [2019],Shah et al [2018] …”

mentioning

confidence: 99%

How Complex is your classification problem? A survey on measuring classification complexity

Lorena,

Garcia,

Lehmann

et al. 2018

Preprint

View full text Add to dashboard Cite

Characteristics extracted from the training datasets of classification problems have proven to be effective predictors in a number of meta-analyses. Among them, measures of classification complexity can be used to estimate the difficulty in separating the data points into their expected classes. Descriptors of the spatial distribution of the data and estimates of the shape and size of the decision boundary are among the known measures for this characterization. This information can support the formulation of new data-driven pre-processing and pattern recognition techniques, which can in turn be focused on challenges highlighted by such characteristics of the problems. This paper surveys and analyzes measures which can be extracted from the training datasets in order to characterize the complexity of the respective classification problems. Their use in recent literature is also reviewed and discussed, allowing to prospect opportunities for future work in the area. Finally, descriptions are given on an R package named Extended Complexity Library (ECoL) that implements a set of complexity measures and is made publicly available.

show abstract

Can Complexity Measures and Instance Hardness Measures Reflect the Actual Complexity of Microarray Data?

Al Hosni,

Starkey

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Analyzing Data Complexity Using Metafeatures for Classification Algorithm Selection

Cited by 5 publications

References 9 publications

Explainable Rules and Heuristics in AI Algorithm Recommendation Approaches—A Systematic Literature Review and Mapping Study

Explainable Rules and Heuristics in AI Algorithm Recommendation Approaches—A Systematic Literature Review and Mapping Study

How Complex is your classification problem? A survey on measuring classification complexity

Can Complexity Measures and Instance Hardness Measures Reflect the Actual Complexity of Microarray Data?

Contact Info

Product

Resources

About