Use of Random forest in the identification of important variables

Lovatti, Betina P.O.; Nascimento, Márcia H.C.; Neto, Álvaro Cunha; Castro, Eustáquio Vinícius Ribeiro de; Filgueiras, Paulo R.

doi:10.1016/j.microc.2018.12.028

Cited by 52 publications

(30 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The RF models were evaluated by the degree of accuracy, using the results of the first strategy (RF model) and the second strategy (FD‐RF model). For each development strategy, the model with more accuracy was chosen for the identification of predictor variables with more contribution in sample discrimination 4 . The identification of predictor variables was carried out using trees with an accuracy level above 70%, owing to the constant presence of these variables in the construction of the best models.…”

Section: Methodsmentioning

confidence: 99%

“…To accomplish this, the RF model of the first strategy and FD-RF model of the second strategy were used. 4 2.1 | 13 C NMR measurements to 40% of the final volume of the solution in deuterated chloroform (CDCl 3 ), containing 0.05 molÁL −1 of III chromium acetylcetonate 3-mL total volume. The instrumental conditions were frequency, 100.51 MHz for the 13 C nucleus; spectral window, 25 510.2 Hz; acquisition time, 1.285 s; standby time, 7 seconds; pulse, 90 (14.2 μs); number of transients, 1000; and decoupling mode, nny.…”

Section: Methodsmentioning

confidence: 99%

“…From the 1 H NMR dataset (20 433 variables) and 13 C NMR dataset (61 606 variables) of the 126 crude oil samples, we developed our RF classification models. 4 All models were obtained with bootstrap to form the training set, called "in bag" (approximately two thirds of samples), and the test set, called "out-of-bag" (approximately one third of samples). In all model development strategies, machine processing time and costs were compared (Table S1).…”

Section: Random Forest Classification Modelmentioning

confidence: 99%

See 2 more Smart Citations

Different strategies for the use of random forest in NMR spectra

Lovatti

Nascimento

Rainha

et al. 2020

Journal of Chemometrics

Self Cite

View full text Add to dashboard Cite

Nuclear magnetic resonance (NMR) can provide a large amount of information about an analyzed sample; however, its spectra contain above 6000 variables, making it difficult for random forest (RF) applications. Reducing the size of the original dataset can minimize this problem. In this paper, we compared RF classification models obtained with full NMR spectral range and from the reduction of NMR variables, using principal component analysis (PCA) and the Fisher discriminant (FD). Then, the variables used in the construction of RF trees were analyzed and identified. Here, we used 1 H and 13 C NMR spectra obtained from 126 petroleum samples and values of their total acidy number (TAN), as measured by ASTM D664, ranging from 0.03 to 4.96 mg KOHÁ g −1 , to distinguish the oil samples from the TAN values. Of two classes that resulted, the first contained 78 samples with TAN values less than, or equal to, 0.3 mg KOHÁ g −1 , while the second contained 48 samples with TAN values higher than 0.3 mg KOHÁ g −1. The 1 H NMR results showed that the combination of FD and RF techniques provided the best accuracy (88%). For 13 C NMR data, the most accurate model was obtained by the association of PCA and RF (84%). The identification of variables used in RF allowed a better understanding of the important chemical data contained in the spectra and the relationship to TAN in petroleum. K E Y W O R D S fisher discriminant, NMR, PCA, random forest, reduction of variables 1 | INTRODUCTION Random forest (RF) is a machine learning method recognized for its efficiency in supervised classification of linear and nonlinear data, especially in the areas of genetic analysis, metabolomics, medical image analysis, food quality control, and crude oil. 1-4 The potential of the RF technique in classification is confirmed when associated with analytical methods that allow the acquisition of a large amount of information, such as chromatography, mass spectrometry, infrared spectroscopy, nuclear magnetic resonance of hydrogen (1 H NMR), and carbon (13 C NMR). 1-4 The NMR technique provides detailed information on functional groups, characterizing the sample at the molecular level; however, the number of variables generated may exceed 65 000, depending on the spectral range analyzed. In

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Random Forest Classification Modelmentioning

confidence: 99%

See 1 more Smart Citation

Different strategies for the use of random forest in NMR spectra

Lovatti

Nascimento

Rainha

et al. 2020

Journal of Chemometrics

Self Cite

View full text Add to dashboard Cite

show abstract

“…sample classes. Then the tree is divided into many branches until an outcome (decision node or lead) is attained [43]. It is challenging to obtain optimal parameters using a Multilayer perceptron; the robustness of SVM is not always good and random forest has overfitting issues [44].…”

Section: Value Of All the Attributesmentioning

confidence: 99%

Realizing a Stacking Generalization Model to Improve the Prediction Accuracy of Major Depressive Disorder in Adults

et al. 2020

View full text Add to dashboard Cite

Major depressive disorder (MDD) is a persistent psychiatric mood disorder that is prevalent from a few weeks to a few months, even for years in the worst cases. It causes sadness, hopelessness in the individuals; sometimes, it forces them to hurt themselves. In severe cases, MDD can even lead to the death of the individual. It is challenging to diagnose MDD as it co-occurs with many other disorders (Co-Morbid) and many other reasons such as mobility, lack of motivation, and cost. The way to diagnose MDD is usually high ended that is challenging for the regular clinicians to diagnose. Therefore, to make their work more comfortable, and to predict MDD at the early stages, we have developed an ensemble-based machine learning model. The data collected has been cleaned with a preprocessing technique, and feature selection are performed using wrapper based methods; moreover, in the final step, a stacking based ensemble learning model is implemented to classify the MDD patients. Furthermore, KNN Imputation is implemented for preprocessing, Random Forest-Based Backward Elimination for feature selection and multi-layer perceptron, SVM and Random Forest as low-level learners in stacking generalization model. The results show that the prediction accuracy of the stacking generalization model is superior to the individual classifiers. INDEX TERMS K-nearest neighbors, major depressive disorder, multilayer perceptron, random forest, random forest-based feature elimination, stacking generalization and support vector machine.

show abstract

“…For the XGBoost model, it generates importance score XG i based on the frequency at which x i is used to segment the study data during node segmentation of trees established [30]. RF calculates the importance score RF i by determining the average contribution of x i on each tree [31]. Calculating the average value of XG i and RF i can reduce the influence of the difference caused by different importance analysis methods.…”

Section: ) Importance Analysismentioning

confidence: 99%

A Cluster-Stacking-Based Approach to Forecasting Seasonal Chlorophyll-a Concentration in Coastal Waters

Jia

Cheng

2020

IEEE Access

View full text Add to dashboard Cite

Prediction of Chlorophyll-a (Chl-a) concentration is significant for marine ecology and environmental protection. This paper presents an integrated approach to forecast seasonal Chl-a concentration in coastal waters. Before modeling, feature construction procedures, such as simplification, combination, and normalization, are conducted to identify the potentially significant features. The feature extraction method based on Random Forest (RF) and eXtreme Gradient BOOSTing (XGBoost) is applied to select relevant variables. Then, we propose a Cluster-stacking-based approach which includes a station-oriented clustering model and a stacking-based regression model. The former model is used to divide the observation stations into several groups, thus partitions the study region into several sub-regions and the study dataset into several subsets according to the corresponding stations. In each subset, single regression models including K-Nearest Neighbor (KNN), Support Vector Regression (SVR), Multi-Layer Perceptron regression (MLP) and XGBoost are established in level 0 space and integrated by RF in level 1 space via stacked generalization. We compare the performance of the Cluster-stacking model with that of Cluster-KNN, Cluster-SVR, Cluster-MLP, Cluster-XGBoost and the regression stacking model without cluster. The model evaluation shows that the Cluster-stacking-based approach outperforms others in forecasting Chl-a concentration with a coefficient of determination (R 2) of 0.848 and a mean absolute error (MAE) of 0.665 ug/l. INDEX TERMS Chlorophyll-a concentration forecast, cluster-stacking-based approach, regression stacking, coastal waters.

show abstract

Use of Random forest in the identification of important variables

Cited by 52 publications

References 37 publications

Different strategies for the use of random forest in NMR spectra

Different strategies for the use of random forest in NMR spectra

Realizing a Stacking Generalization Model to Improve the Prediction Accuracy of Major Depressive Disorder in Adults

A Cluster-Stacking-Based Approach to Forecasting Seasonal Chlorophyll-a Concentration in Coastal Waters

Contact Info

Product

Resources

About