Combination of Feature Selection and CatBoost for Prediction: The First Application to the Estimation of Aboveground Biomass

Abstract

Increasing numbers of explanatory variables tend to result in information redundancy and “dimensional disaster” in the quantitative remote sensing of forest aboveground biomass (AGB). Feature selection of model factors is an effective method for improving the accuracy of AGB estimates. Machine learning algorithms are also widely used in AGB estimation, although little research has addressed the use of the categorical boosting algorithm (CatBoost) for AGB estimation. Both feature selection and regression for AGB estimation models are typically performed with the same machine learning algorithm, but there is no evidence to suggest that this is the best method. Therefore, the present study focuses on evaluating the performance of the CatBoost algorithm for AGB estimation and comparing the performance of different combinations of feature selection methods and machine learning algorithms. AGB estimation models of four forest types were developed based on Landsat OLI data using three feature selection methods (recursive feature elimination (RFE), variable selection using random forests (VSURF), and least absolute shrinkage and selection operator (LASSO)) and three machine learning algorithms (random forest regression (RFR), extreme gradient boosting (XGBoost), and categorical boosting (CatBoost)). Feature selection had a significant influence on AGB estimation. RFE preserved the most informative features for AGB estimation and was superior to VSURF and LASSO. In addition, CatBoost improved the accuracy of the AGB estimation models compared with RFR and XGBoost. AGB estimation models using RFE for feature selection and CatBoost as the regression algorithm achieved the highest accuracy, with root mean square errors (RMSEs) of 26.54 Mg/ha for coniferous forest, 24.67 Mg/ha for broad-leaved forest, 22.62 Mg/ha for mixed forests, and 25.77 Mg/ha for all forests. The combination of RFE and CatBoost had better performance than the VSURF–RFR combination in which random forests were used for both feature selection and regression, indicating that feature selection and regression performed by a single machine learning algorithm may not always ensure optimal AGB estimation. It is promising to extending the application of new machine learning algorithms and feature selection methods to improve the accuracy of AGB estimates.

Building Tree Allometry Relationships Based on TLS Point Clouds and Machine Learning Regression

“…In fact, small changes in the learning sample can cause dramatic changes in the built tree derived from individual tree-based models, and so the estimated results can be unstable and inaccurate. This is the reason why most recent studies have adopted bagging and boosting ensemble algorithms [25,26].…”

“…Machine learning methods have been widely applied to develop AGB prediction models from different remote sensing data sources such as optical satellite imagery, UAV stereo-imagery, airborne hyperspectral images, ALS (airborne laser scanning), and spaceborne SAR [15,16,25,26,36,37].…”

Comparing the performance of machine learning algorithms for remote and in situ estimations of chlorophyll‐a content: A case study in the Tri An Reservoir, Vietnam

“…In this study, the CB and the XGB are well‐known ML algorithms and successfully applied for the estimation of various biophysical parameters (Ha et al, 2021b; Luo et al, 2021; Naghibi et al, 2020; Pham et al, 2020). However, the implementation of these models for Chl‐a estimation in a highly eutrophic reservoir is limited; therefore, our results might suggest novel findings for model selection and their performance as well.…”

Predictive Innovative Methods for Aquatic Heavy Metals Pollution Based on Bioindicators in Support of Blue Economy in the Danube River Basin

“…During the past six years, the random forest usage has steadily increased. By using Google Trends service, it was possible to collect historic data that was used to visualize the search interest for five prediction related terms: (a) linear regression-one of the most used prediction algorithms, (b) random forest, (c) XGBoost-a regularizing gradient boosting algorithm enforced by a strong community of data scientists [63], (d) CatBoost-one of the newest gradient boosting prediction algorithms [64], (e) prediction model-a generic term used to describe any of the previous mentioned algorithms (Figure 3). As it can be observed in Figure 3, the search interest value for the 'linear regression' term is the highest (which is to be expected due to its age and worldwide popularity).…”