Ensemble partial least squares regression for descriptor selection, outlier detection, applicability domain assessment, and ensemble modeling in QSAR/QSPR modeling

Abstract

In QSAR/QSPR modeling, building an accurate partial least squares (PLS) model usually involves descriptor selection, outlier detection, applicability domain assessment, nonlinear relationship, and model stability problems. In the present study, we presented an ensemble PLS (EnPLS) method for solving these modeling tasks under a unified methodology framework. EnPLS aims at developing a consistent algorithmic framework by means of the idea of ensemble learning and statistical distribution. The approach exploits the fact that the distribution of PLS model coefficients provides a mechanism for ranking and interpreting the effects of variables, whereas the distribution of prediction errors provides a mechanism for differentiating the outliers from normal samples and assessing the applicability domain of models. The use of statistics of these distributions, namely, mean/median value and standard deviation, inherently provides a feasible way to effectively describe the information contained by the original samples. Furthermore, ensemble modeling and prediction based on several cross-predictive PLS models could effectively improve the model prediction performance and increase the model stability to a certain extent. The aqueous solubility data are used to demonstrate the ability of our proposed EnPLS method in solving various modeling tasks such as descriptor selection, outlier detection, applicability domain assessment, performance improvement, and model stability. Finally, a freely available R package implementing EnPLS is developed to facilitate the use of chemists and pharmacologists. The R package is freely available at https://github.com/wind22zhu/enpls1.2. KEYWORDS applicability domain assessment, ensemble learning, outlier detection, partial least squares (PLS), QSAR/ QSPR, variable selection 1 | INTRODUCTIONChemometrics aims at solving chemical problems using statistical or informatics methods. There is a great deal of previous successful research in this filed, revolving around topics such as quantitative structure-activity relationships (QSARs), quantitative structure-property relationships (QSPRs), multivariate calibration, reaction design, and drug design. The QSAR/QSPR methodology aims at finding a model, which allows for correlating the activities to structures within a family of compounds.

A novel method to guide classification of para swimmers with limb deficiency

“…The “enpls” argument in the enpls package was used to conduct partial least squares regression on 500 Monte Carlo experiments with a sampling ratio of 0.8. As the determined effects of independent variables to any model are dependent upon the cases included in analyses, ensemble learning methods may improve prediction accuracy and stability of regression models by exploiting the statistical distribution of variable coefficients and prediction errors . K‐fold cross‐validation was used to examine the stability of the ensemble partial least squares model using the “cv.enpls()” argument …”

“…Variable coefficients and importance scores were calculated using the “fs.enpls()” function in the enpls package . The vector of regression coefficients across multiple Monte Carlo experiments were used to indicate predictor importance because it is a single measure of association between the predictor variables and the response .…”

“…Variable coefficients and importance scores were calculated using the “fs.enpls()” function in the enpls package . The vector of regression coefficients across multiple Monte Carlo experiments were used to indicate predictor importance because it is a single measure of association between the predictor variables and the response . The variable importance score was calculated as the mean of the variable coefficient divided by the standard deviation of the variable coefficient for the 500 Monte Carlo experiments .…”

“…The vector of regression coefficients across multiple Monte Carlo experiments were used to indicate predictor importance because it is a single measure of association between the predictor variables and the response . The variable importance score was calculated as the mean of the variable coefficient divided by the standard deviation of the variable coefficient for the 500 Monte Carlo experiments . Thus, a larger variable importance score indicates a larger and more consistent regression coefficient across the 500 Monte Carlo experiments.…”

The impact of limb deficiency impairment on Para swimming performance

“…The advantage of this method is that while the independent variables may be collinear the derived x-components used in regression will be independent of one another (Mevik & Wehrens, 2007). The ensemble learning method is also advantageous as it exploits the distribution in prediction errors and regression coefficients within a dataset to improve prediction accuracy and stability of regression (Cao et al, 2017).…”