On Incremental Learning for Gradient Boosting Decision Trees

Zhang, Chongsheng; Zhang, Yuwei; Shi, Xianjin; Almpanidis, George; Fan, Gaojuan; Shen, Xiajiong

doi:10.1007/s11063-019-09999-3

Cited by 67 publications

(30 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The extreme gradient boosting (XGBoost) method is a kind of gradient boosting decision tree (GBDT) [50] technique, which can be used for both classification and regression problems. As described in [51], gradient boosting is an ensemble learning method that combines a set of weak classifiers f i (x) to form a strong classifier F(x).…”

Section: Extreme Gradient Boosting (Xgboost)mentioning

confidence: 99%

See 1 more Smart Citation

IoT Botnet Attack Detection Based on Optimized Extreme Gradient Boosting and Feature Selection

Alqahtani

Mathkour

Ismail

2020

Sensors

View full text Add to dashboard Cite

Nowadays, Internet of Things (IoT) technology has various network applications and has attracted the interest of many research and industrial communities. Particularly, the number of vulnerable or unprotected IoT devices has drastically increased, along with the amount of suspicious activity, such as IoT botnet and large-scale cyber-attacks. In order to address this security issue, researchers have deployed machine and deep learning methods to detect attacks targeting compromised IoT devices. Despite these efforts, developing an efficient and effective attack detection approach for resource-constrained IoT devices remains a challenging task for the security research community. In this paper, we propose an efficient and effective IoT botnet attack detection approach. The proposed approach relies on a Fisher-score-based feature selection method along with a genetic-based extreme gradient boosting (GXGBoost) model in order to determine the most relevant features and to detect IoT botnet attacks. The Fisher score is a representative filter-based feature selection method used to determine significant features and discard irrelevant features through the minimization of intra-class distance and the maximization of inter-class distance. On the other hand, GXGBoost is an optimal and effective model, used to classify the IoT botnet attacks. Several experiments were conducted on a public botnet dataset of IoT devices. The evaluation results obtained using holdout and 10-fold cross-validation techniques showed that the proposed approach had a high detection rate using only three out of the 115 data traffic features and improved the overall performance of the IoT botnet attack detection process.

show abstract

Section: Extreme Gradient Boosting (Xgboost)mentioning

confidence: 99%

“…The gradient boosting tries to correct the residuals of all the weak learners by adding new weak learners [50]. In the end, multiple learners are added together for the final prediction and the accuracy is higher than for a single learner.…”

Section: Extreme Gradient Boosting (Xgboost)mentioning

confidence: 99%

IoT Botnet Attack Detection Based on Optimized Extreme Gradient Boosting and Feature Selection

Alqahtani

Mathkour

Ismail

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…by evolving the DT into a Gradient Boosting) and speed-up the re-training procedure (i.e. by exploiting incremental learning strategies [37]).…”

Section: Discussionmentioning

confidence: 99%

Machine Learning in Capital Markets: Decision Support System for Outcome Analysis

Rosati

Romeo

Goday³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Decision support systems using Artificial Intelligence in the context of financial services includes different application ranging from investment advice to financial trading. The analysis of order flow provides many challenges that can be addressed by Machine Learning (ML) techniques in order to determine an optimal dynamic trading strategy. The first step in this direction is represented by the outcome analysis of order flow: the model should identify strong predictors that determine a positive/negative outcome. The aim of this work is the proposal of a closed-loop ML approach based on decision tree (DT) model to perform outcome analysis on financial trading data. The overall approach is integrated in a Decision Support System for Outcome Analysis (DSS-OA). Taking into account the model complexity, the DT algorithm enables to generate explanations that allow the user to understand (i) how this outcome is reached (decision rules) and (ii) the most discriminative outcome predictors (feature importance). The closed-loop approach allows the users to interact directly with the proposed DSS-OA by retraining the algorithm with the goal to a finer-grained outcome analysis. The experimental results and comparisons demonstrated high-interpretability and predictive performance of the proposed DSS-OA by providing a valid and fast system for outcome analysis on financial trading data. Moreover, the Proof of Concept evaluation demonstrated the impact of the proposed DSS-OA in the outcome analysis scenario.

show abstract

“…Gradient boosting (GB) is an ensemble learning technique, used for classification and regression problems, proposed by Friedman [51,52]. It can produce an effective model consisting of weak learners, usually decision trees.…”

Section: Methodsmentioning

confidence: 99%

A Genetic-Based Extreme Gradient Boosting Model for Detecting Intrusions in Wireless Sensor Networks

Alqahtani

Gumaei

Mathkour

et al. 2019

Sensors

View full text Add to dashboard Cite

An Intrusion detection system is an essential security tool for protecting services and infrastructures of wireless sensor networks from unseen and unpredictable attacks. Few works of machine learning have been proposed for intrusion detection in wireless sensor networks and that have achieved reasonable results. However, these works still need to be more accurate and efficient against imbalanced data problems in network traffic. In this paper, we proposed a new model to detect intrusion attacks based on a genetic algorithm and an extreme gradient boosting (XGBoot) classifier, called GXGBoost model. The latter is a gradient boosting model designed for improving the performance of traditional models to detect minority classes of attacks in the highly imbalanced data traffic of wireless sensor networks. A set of experiments were conducted on wireless sensor network-detection system (WSN-DS) dataset using holdout and 10 fold cross validation techniques. The results of 10 fold cross validation tests revealed that the proposed approach outperformed the state-of-the-art approaches and other ensemble learning classifiers with high detection rates of 98.2%, 92.9%, 98.9%, and 99.5% for flooding, scheduling, grayhole, and blackhole attacks, respectively, in addition to 99.9% for normal traffic.

show abstract

On Incremental Learning for Gradient Boosting Decision Trees

Cited by 67 publications

References 16 publications

IoT Botnet Attack Detection Based on Optimized Extreme Gradient Boosting and Feature Selection

IoT Botnet Attack Detection Based on Optimized Extreme Gradient Boosting and Feature Selection

Machine Learning in Capital Markets: Decision Support System for Outcome Analysis

A Genetic-Based Extreme Gradient Boosting Model for Detecting Intrusions in Wireless Sensor Networks

Contact Info

Product

Resources

About