Multi-objective hyperparameter tuning and feature selection using filter ensembles

Binder, Martin; Moosbauer, Julia; Thomas, Janek; Bischl, Bernd

doi:10.1145/3377930.3389815

Cited by 34 publications

(28 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The latter can either be a single value or an interval of values. We define a counterfactual explanation x for an observation x * as a data point fulfilling the following: (1) its prediction f (x ) is close to the desired outcome set Y , (2) it is close to x * in the X space, (3) it differs from x * only in a few features, and (4) it is a plausible data point according to the probability distribution P X . For classification models, we assume thatf returns the probability for a user-selected class and Y has to be the desired probability (range).…”

Section: Multi-objective Counterfactualsmentioning

confidence: 99%

“…The complete code of the algorithm and the code to reproduce the experiments and results of this paper are available at https://github.com/susanne-207/moc. The implementation of MOC is based on our implementation of [19], which we also used for [3]. We will provide an open source R library with our implementation of the method based on the iml package [23].…”

Section: Electronic Submissionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Objective Counterfactual Explanations

Dandl

Molnar

Binder

et al. 2020

Lecture Notes in Computer Science

Self Cite

170

125

View full text Add to dashboard Cite

Counterfactual explanations are one of the most popular methods to make predictions of black box machine learning models interpretable by providing explanations in the form of 'what-if scenarios'. Most current approaches optimize a collapsed, weighted sum of multiple objectives, which are naturally difficult to balance a-priori. We propose the Multi-Objective Counterfactuals (MOC) method, which translates the counterfactual search into a multi-objective optimization problem. Our approach not only returns a diverse set of counterfactuals with different trade-offs between the proposed objectives, but also maintains diversity in feature space. This enables a more detailed post-hoc analysis to facilitate better understanding and also more options for actionable user responses to change the predicted outcome. Our approach is also model-agnostic and works for numerical and categorical input features. We show the usefulness of MOC in concrete cases and compare our approach with state-of-the-art methods for counterfactual explanations.

show abstract

Section: Multi-objective Counterfactualsmentioning

confidence: 99%

Section: Electronic Submissionmentioning

confidence: 99%

Multi-Objective Counterfactual Explanations

Dandl

Molnar

Binder

et al. 2020

Lecture Notes in Computer Science

Self Cite

170

125

View full text Add to dashboard Cite

show abstract

“…Hyperparameters were tuned using model-based optimization (MBO) within a nested spatial cross-validation (CV) [50][51][52]. In MBO, first, n hyperparameter settings are randomly chosen from a user-defined search space.…”

Section: Hyperparameter Optimizationmentioning

confidence: 99%

“…To optimize the number of features used for model fitting, the percentage of features was added as a hyperparameter during the optimization stage [51]. For PCA, the number of principal components was tuned.…”

Section: Hyperparameter Optimizationmentioning

confidence: 99%

Monitoring Forest Health Using Hyperspectral Imagery: Does Feature Selection Improve the Performance of Machine-Learning Techniques?

et al. 2021

Self Cite

View full text Add to dashboard Cite

This study analyzed highly correlated, feature-rich datasets from hyperspectral remote sensing data using multiple statistical and machine-learning methods. The effect of filter-based feature selection methods on predictive performance was compared. In addition, the effect of multiple expert-based and data-driven feature sets, derived from the reflectance data, was investigated. Defoliation of trees (%), derived from in situ measurements from fall 2016, was modeled as a function of reflectance. Variable importance was assessed using permutation-based feature importance. Overall, the support vector machine (SVM) outperformed other algorithms, such as random forest (RF), extreme gradient boosting (XGBoost), and lasso (L1) and ridge (L2) regressions by at least three percentage points. The combination of certain feature sets showed small increases in predictive performance, while no substantial differences between individual feature sets were observed. For some combinations of learners and feature sets, filter methods achieved better predictive performances than using no feature selection. Ensemble filters did not have a substantial impact on performance. The most important features were located around the red edge. Additional features in the near-infrared region (800–1000 nm) were also essential to achieve the overall best performances. Filter methods have the potential to be helpful in high-dimensional situations and are able to improve the interpretation of feature effects in fitted models, which is an essential constraint in environmental modeling studies. Nevertheless, more training data and replication in similar benchmarking studies are needed to be able to generalize the results.

show abstract

“…To optimize the number of features used for model fitting, the percentage of features was added as a hyperparameter during the optimization stage ( [39]).…”

Section: Benchmarking Design 1) Algorithmsmentioning

confidence: 99%

Monitoring forest health using hyperspectral imagery: Does feature selection improve the performance of machine-learning techniques?

Schratz¹,

Muenchow²,

Iturritxa³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

This study analyzed highly-correlated, feature-rich datasets from hyperspectral remote sensing data using multiple machine and statistical-learning methods. The effect of filter-based feature-selection methods on predictive performance was compared. Also, the effect of multiple expert-based and data-driven feature sets, derived from the reflectance data, was investigated. Defoliation of trees (%) was modeled as a function of reflectance, and variable importance was assessed using permutation-based feature importance. Overall support vector machine (SVM) outperformed others such as random forest (RF), extreme gradient boosting (XGBoost), lasso (L1) and ridge (L2) regression by at least three percentage points. The combination of certain feature sets showed small increases in predictive performance while no substantial differences between individual feature sets were observed. For some combinations of learners and feature sets, filter methods achieved better predictive performances than the unfiltered feature sets, while ensemble filters did not have a substantial impact on performance. Permutation-based feature importance estimated features around the red edge to be most important for the models. However, the presence of features in the near-infrared region (800 nm - 1000 nm) was essential to achieve the best performances. More training data and replication in similar benchmarking studies is needed for more generalizable conclusions. Filter methods have the potential to be helpful in high-dimensional situations and are able to improve the interpretation of feature effects in fitted models, which is an essential constraint in environmental modeling studies.

show abstract

Multi-objective hyperparameter tuning and feature selection using filter ensembles

Cited by 34 publications

References 29 publications

Multi-Objective Counterfactual Explanations

Multi-Objective Counterfactual Explanations

Monitoring Forest Health Using Hyperspectral Imagery: Does Feature Selection Improve the Performance of Machine-Learning Techniques?

Monitoring forest health using hyperspectral imagery: Does feature selection improve the performance of machine-learning techniques?

Contact Info

Product

Resources

About