CatBoost for Big Data: an Interdisciplinary Review

Abstract

Gradient Boosted Decision Trees (GBDT's) are a powerful tool for classification and regression tasks in Big Data, Researchers should be familiar with the strengths and weaknesses of current implementations of GBDT's in order to use them effectively and make successful contributions. CatBoost is a member of the family of GBDT machine learning ensemble techniques. Since its debut in late 2018, researchers have ellCcessfully used CatBoost for machine learning studies involving Big Data. We take this opportunity to review recent research on CatBoost as it relates to Big Data, and learn best practices from studies that .55 CatBoost in a positive light, as well as studies where CatBoost does not outshine other techniques, since we can learn lessons from both types of scenarios. Furthermore, as a Decision Tree based algorithm, CatBoost is well-suited to machine learning tasks involving categorical, heterogeneous data. Recent work across multiple disciplines illustrates CatBoost's effectiveness and shortcomings in classification and regression tasks. Another important issue we expose in literature on CatBoost is its sensitivity to hyper-parameters and the importance of hyper-parameter tuning. One contribution we make is to take an interdisciplinary approach to cover studies related to CatBoost in a single work. This provides researchers an in-depth understanding to help clarify proper application of CatBoost in solving problems. To the best of our knowledge, this is the first survey that studies all works related to CatBoost in a single publication.

Analysis of Ensembled Learning using Classical and Quantum Boosting Approaches for Diabetes Mellitus Prediction

“…It is a high-performance open source library for gradient boosting on decision tree. It gives outstanding output, and it performs quickly by the help of GPU, even on a large dataset [23].…”

“…Gradient boosting now takes an additive pattern that recursively builds a greedy sequence of interpolation F t of a given loss function L(y i , F t ). Assuming the function f (x),we can enhance estimate of y i by finding another function [23].…”

Forecasting fertility demographic indicators: inertial method versus machine learning method

“…Второй метод, рассмотренный в статье -это метод машинного обучения на основе градиентного бустинга (Catboost), который позволяет создавать множество алгоритмов (деревьев решения), которые, в свою очередь, способны научиться принимать решения и строить прогнозы на основе данных (Brink et al, 2016;Hancock and Khoshgoftaar, 2020). Машинное обучение -сравнительно новый метод, при этом для демографической науки он редко применялся ранее (Соловьев и Соловьев, 2018, с.…”

BioAutoML: Democratizing Machine Learning in Life Sciences

“…• Given a set of selected features, recommend the ML algorithm(s) able to induce the best predictive model, which can be a set of algorithms, each one inducing a model, and combine these models into an ensemble (P ml ), recommending the best algorithm. Ensemble methods can boost the performance of simple classifiers (e.g., using multiple prediction models for solving the same problem) and have proven their effectiveness in bioinformatics (LIU et al, 2020;HANCOCK;KHOSHGOFTAAR, 2020;HE et al, 2022).…”

“…We chose these ML algorithms because they have good predictive performance and induce interpretable predictive models, allowing the understanding of the internal decision-making process (BONIDIA et al, 2020a). The algorithms are widely adopted in the bioinformatics literature (LIU et al, 2020;HANCOCK;KHOSHGOFTAAR, 2020;HE et al, 2022).…”