Comparison and improvement of the predictability and interpretability with ensemble learning models in QSPR applications

Abstract

Ensemble learning helps improve machine learning results by combining several models and allows the production of better predictive performance compared to a single model. It also benefits and accelerates the researches in quantitative structure-activity relationship (QSAR) and quantitative structure-property relationship (QSPR). With the growing number of ensemble learning models such as random forest, the effectiveness of QSAR/QSPR will be limited by the machine's inability to interpret the predictions to researchers. In fact, many implementations of ensemble learning models are able to quantify the overall magnitude of each feature. For example, feature importance allows us to assess the relative importance of features and to interpret the predictions. However, different ensemble learning methods or implementations may lead to different feature selections for interpretation. In this paper, we compared the predictability and interpretability of four typical well-established ensemble learning models (Random forest, extreme randomized trees, adaptive boosting and gradient boosting) for regression and binary classification modeling tasks. Then, the blending methods were built by summarizing four different ensemble learning methods. The blending method led to better performance and a unification interpretation by summarizing individual predictions from different learning models. The important features of two case studies which gave us some valuable information to compound properties were discussed in detail in this report. QSPR modeling with interpretable machine learning techniques can move the chemical design forward to work more efficiently, confirm hypothesis and establish knowledge for better results.

Health Condition Estimation of Bearings with Multiple Faults by a Composite Learning-Based Approach

“…To study the effect of training with a different tier-one algorithm (meta-learner) on the overall results, the decision trees were introduced in the architecture as an alternative to the SVM. Decision trees are a well-known method and are fast to train [ 42 ]. The result from this modified blended ensemble model (ECNN-DT) was also better than the individual base models.…”

“…The base inducers can be combined using methods such as stack generalization/blending [ 39 ], using different algebraic functions [ 40 ], non-linear combination methods (for instance, Dempster–Shafer belief methods) [ 41 ]. Stacked generalization is an ensemble learning approach that applies a meta-learner and out-of-fold prediction of the training set to detect the best way of combining the base models’ outputs [ 42 ]. A variant of stacked generalization is blending.…”

“…A variant of stacked generalization is blending. Blending reduces information leaks and it is more straightforward [ 42 ]. In blending, the predictions from each of the tier-zero models are fed as training data to the meta-learner.…”

Progress on open chemoinformatic tools for expanding and exploring the chemical space

“…Overall, a critical and controversial point of chemical descriptors is their interpretability and physical meaning. In predictive models, it is open for discussion if the descriptors do not only show how a good statistical association between the chemical structure and the property (e.g., biological activity) of interest but if the descriptors can actually explain or contribute to the causality of the activity as encoded by the chemical descriptors [ 58 , 59 ].…”

Prediction and Chemical Interpretation of Singlet-Oxygen-Scavenging Activity of Small Molecule Compounds by Using Machine Learning

“…The numerical values obtained from computational chemistry and Morgan fingerprinting [ 15 ] were used as the input data. To obtain the interpretability of the machine learning model, regression coefficients were obtained from the linear model, and the feature importance was obtained from the prediction model based on the decision tree [ 16 ]. To verify the interpretability of the model, we attempted to explain the reaction mechanism [ 17 ] of singlet oxygen scavenging from the feature importance.…”