Explaining Hyperparameter Optimization via Partial Dependence Plots

Abstract

Automated hyperparameter optimization (HPO) can support practitioners to obtain peak performance in machine learning models. However, there is often a lack of valuable insights into the effects of different hyperparameters on the final model performance. This lack of explainability makes it difficult to trust and understand the automated HPO process and its results. We suggest using interpretable machine learning (IML) to gain insights from the experimental data obtained during HPO with Bayesian optimization (BO). BO tends to focus on promising regions with potential high-performance configurations and thus induces a sampling bias. Hence, many IML techniques, such as the partial dependence plot (PDP), carry the risk of generating biased interpretations. By leveraging the posterior uncertainty of the BO surrogate model, we introduce a variant of the PDP with estimated confidence bands. We propose to partition the hyperparameter space to obtain more confident and reliable PDPs in relevant sub-regions. In an experimental study, we provide quantitative evidence for the increased quality of the PDPs within sub-regions.

Puzzle out Machine Learning Model-Explaining Disintegration Process in ODTs

“…ML models are mostly black-box in nature, but attempts can be made to explain their ways of generating predictions. Our workflow ( Figure 1 ) utilizes two post-hoc model agnostic methods for model interpretation: the SHapley Additive exPlanations (SHAP) method by Lundberg et al [ 64 ], which helps develop reasoning behind individual predictions of model, and partial dependency plots (PDP) [ 65 ], which are used to represent global relationships between input and output variables. The implementation of both methods was made using an in-house-developed Python wrapper script [ 66 ].…”

A random forest-based analysis of cassava mosaic disease-related factors affecting the on-farm livelihoods of cassava farmers

“…These plots visualize the marginal effect of a feature on the predicted outcomes of the model, providing valuable insights into how the model is making its predictions. These plots typically show the relationship between a specific feature and the predicted outcome of the model, with the feature on the x-axis and the predicted outcome on the y-axis (Inouye et al, 2020;Moosbauer et al, 2021).…”