Constructing Regression Models with High Prediction Accuracy and Interpretability Based on Decision Tree and Random Forests

Abstract

Models for predicting properties/activities of materials based on machine learning can lead to the discovery of new mechanisms underlying properties/activities of materials. However, methods for constructing models that exhibit both high prediction accuracy and interpretability remain a work in progress because the prediction accuracy and interpretability exhibit a trade-off relationship. In this study, we propose a new model-construction method that combines decision tree (DT) with random forests (RF); which we therefore call DT-RF. In DT-RF, the datasets to be analyzed are divided by a DT model, and RF models are constructed for each subdataset. This enables global interpretation of the data based on the DT model, while the RT models improve the prediction accuracy and enable local interpretations. Case studies were performed using three datasets, namely, those containing data on the boiling point of compounds, their water solubility, and the transition temperature of inorganic superconductors. We examined the proposed method in terms of its validity, prediction accuracy, and interpretability.

Defect rate prediction and failure‐cause diagnosis in a mass‐production process for precision electric components

“…However, it may differ among high, medium and low y‐values. Shimizu and Kaneko proposed a hybrid model of a decision tree (DT) and RF, in which the importance of the RF is calculated for each leaf node of the DT model, with DT providing a global interpretation of the entire dataset and RF providing a local interpretation for each cluster 33 …”

“…Shimizu and Kaneko proposed a hybrid model of a decision tree (DT) and RF, in which the importance of the RF is calculated for each leaf node of the DT model, with DT providing a global interpretation of the entire dataset and RF providing a local interpretation for each cluster. 33 In this study, the number of defects that occur during the mass production of precision electrical components by a Japanese manufacturer should be decreased. The process consists of 11 chemical and physical unit operations.…”

Genetic Algorithm-Based Partial Least-Squares with Only the First Component for Model Interpretation

“…Although this importance is calculated considering the entire value of y , the importance of each X can vary depending on whether the y value is high, middle, or low. Shimizu and Kaneko proposed the DT and RF hybrid model that successfully interpreted global and local relationships between y and X …”

“…Shimizu and Kaneko proposed the DT and RF hybrid model that successfully interpreted global and local relationships between y and X. 11 In addition to RF, there are other contribution indexes such as the local interpretable model-agnostic explanations (LIME) 12 and Shapley additive explanations (SHAP) 13 that can be combined with any regression analysis methods. In LIME and SHAP, by obtaining an approximation of the shape of the model at a certain sample point, the slope of X with respect to y around that sample point is obtained.…”

Cross‐validated permutation feature importance considering correlation between features

“…The feature importance of x is calculated considering the entire value of y but it can be different when the y value is high, medium, or low. Shimizu and Kaneko (2021) proposed a decision tree (DT) and RF hybrid model where the importance of RF was calculated for each leaf node in the DT model, and thus, DT provided a global interpretation of the entire dataset, and RF provided local interpretations for each cluster. 27 PFI can be used universally with various classification and regression methods such as scikit-learn 28 and also, can be used conveniently.…”