A methodology for the design of experiments in computational intelligence with multiple regression models

Abstract

The design of experiments and the validation of the results achieved with them are vital in any research study. This paper focuses on the use of different Machine Learning approaches for regression tasks in the field of Computational Intelligence and especially on a correct comparison between the different results provided for different methods, as those techniques are complex systems that require further study to be fully understood. A methodology commonly accepted in Computational intelligence is implemented in an R package called RRegrs. This package includes ten simple and complex regression models to carry out predictive modeling using Machine Learning and well-known regression algorithms. The framework for experimental design presented herein is evaluated and validated against RRegrs. Our results are different for three out of five state-of-the-art simple datasets and it can be stated that the selection of the best model according to our proposal is statistically significant and relevant. It is of relevance to use a statistical approach to indicate whether the differences are statistically significant using this kind of algorithms. Furthermore, our results with three real complex datasets report different best models than with the previously published methodology. Our final goal is to provide a complete methodology for the use of different steps in order to compare the results obtained in Computational Intelligence problems, as well as from other fields, such as for bioinformatics, cheminformatics, etc., given that our proposal is open and modifiable.

Avoiding the Inherent Limitations in Datasets Used for Measuring Aesthetics When Using a Machine Learning Approach

“…The design of our experiments was based on a novel methodology for the development of experimental designs in regression problems with multiple machine learning regression algorithms [32]. For each model described above, the optimal set of parameters was sought using hyperparameter optimisation.…”

Decrypting Strong and Weak Single-Walled Carbon Nanotubes Interactions with Mitochondrial Voltage-Dependent Anion Channels Using Molecular Docking and Perturbation Theory

“…The Pool dataset was used with the RRegrs methodology 33 , 37 taking into account all the SWCNT nanodescriptos (V00–V17) to find the best regression model that predicts FEB values for the SWCNT-VDAC interactions. The initial dataset was normalized using R scripts.…”

“…The following regression methods were tested: Multiple Linear regression (LM), Generalized Linear Model with Stepwise Feature Selection (GLM), Lasso regression (Lasso), Partial Least Squares Regression (PLS), Elastic Net regression (ENET), Neural Networks regression (NN), Random Forest (RF), and Random Forest Recursive Feature Elimination (RF-RFE) 33 . Standard RRegrs parameters and methodology were applied: dataset automatically devided by RRegrs using 10 splits (train and test subsets) 33 . The selection of the best regression models used R 2 test (regression coefficient for test subset) and RMSE test (root-mean-square errors for test subset) values Figshare 48 (Fig.…”

“…Examples of these techniques are: Linear Discriminant Analysis (LDA), Neural Networks (NN), Random Forest (RF), etc . 33 – 38 . Last, we should note that, NQSAR/NQSBR algorithms should have the ability of feature selection (FS) to exclude redundant nanodescriptors before NQSAR model building 39 , 40 .…”

“…To verify potential errors in the linear relationship hypothesis on SWCNT nanodescriptors/VDAC channel nanotoxicity, new non-linear classification and regression models were proposed. These non-linear PT-NQSBR models are based on machine learning algorithms implemented on Weka and RRegrs (R package) 33 – 38 . The present work could pave the way for the use of chemo-informatics tools based on SWCNT-ligand and mitotarget docking interactions for making regulatory decisions in Nanotoxicology, allowing the prediction of potential human health impact and environment risks.…”