A fast network intrusion detection system using adaptive synthetic oversampling and LightGBM

Liu, Jingmei; Gao, Yuanbo; Hu, Fengjie

doi:10.1016/j.cose.2021.102289

Cited by 140 publications

(58 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…But LightGBM has more advantages. For example, LightGBM runs 10 times faster than XGBoost while taking only 1/6 memory usage of compared to XGBoost with better accuracy [17].…”

Section: Methodsmentioning

confidence: 99%

Financial Crisis Prediction in Chinese Real Estate Industry from Cash Flow Perspective Based on Machine Learning

Ye¹

2022

Proceedings of the 2022 7th International Conference on Financial Innovation and Economic Development (ICFIED 2022)

View full text Add to dashboard Cite

This paper is aims to establish a financial crisis prediction model in real estate industry. The data was acquired from financial statements of 125 listed real estate companies in China from 2013 to 2017 and listed companies marked with ST or ST* are regarded as enterprises in financial crisis. Since the financial crisis of Chinese real estate enterprises is mostly due to the rupture of cash flow chain and inability to repay debts, as a result, 18 features are selected from four dimensions of operational risk, investment risk, financing risk and capital chain risk based on cash flow perspective. This paper imputes the missing values by K-nearest Neighbor (KNN) imputation method and oversampling for imbalanced dataset using Synthetic Minority Oversampling Technique (SMOTE) method. After that, Light Gradient Boosting Machine (LightGBM) algorithm is used to establish financial crisis prediction model, and the accuracy of the model reached 96%. To illustrate the key factors, this paper ranks the importance of each feature by LightGBM classifier, it can be concluded that the financing risk is very important for enterprises into financial crisis and the project investment in the real estate industry should be treated with more caution. This paper innovatively uses method of machine learning to establish financial crisis prediction model for the real estate industry by cash flow features, which is more in line with the actual situation of the real estate industry.

show abstract

“…But LightGBM has more advantages. For example, LightGBM runs 10 times faster than XGBoost while taking only 1/6 memory usage of compared to XGBoost with better accuracy [17].…”

Section: Methodsmentioning

confidence: 99%

Financial Crisis Prediction in Chinese Real Estate Industry from Cash Flow Perspective Based on Machine Learning

Ye¹

2022

Proceedings of the 2022 7th International Conference on Financial Innovation and Economic Development (ICFIED 2022)

View full text Add to dashboard Cite

show abstract

“…Light Gradient Boosting Machine (LGB) was originally developed by researchers at Microsoft and Peking University to solve the efficiency and scalability problems of GBDT and XGBoost when applied to high-dimensional input features and large data volume problems (Wen, Xie, Wu, & Jiang, 2021). The core concepts of LGB are histogram algorithm, leaf growth strategy with depth limitations, support for category features, histogram feature optimization, multithreading optimization, and cache hit ratio optimization (N. N. Wang, Zhang, Ren, Pang, & Wang, 2021).The algorithm bins the original continuous eigenvalues and uses these bins to build the model, and the histogram greatly reduces the time consumption of the split point selection and improves the training and prediction efficiency of the model (Liu, Gao, & Hu, 2021). With a supervised training set…”

Section: Classification Modelmentioning

confidence: 99%

Research on non-destructive testing of hotpot oil quality by fluorescence hyperspectral technology combined with machine learning

Zou

WU²,

Wang³

et al. 2023

Spectrochimica Acta Part A: Molecular and Biomolecular Spectros

View full text Add to dashboard Cite

“…The proposed method has shown very successful results in detecting and classifying attacks. Liu et al [12] proposed a network intrusion detection system based on adaptive synthetic (ADASYN) oversampling technology and LightGBM. Data imbalance was also discussed in the study.…”

Section: Related Workmentioning

confidence: 99%

STLGBM-DDS: An Efficient Data Balanced DoS Detection System for Wireless Sensor Networks on Big Data Environment

Dener

Orman

2022

IEEE Access

View full text Add to dashboard Cite

Wireless Sensor Networks(WSNs) are vulnerable to a variety of unique security risks and threats in their data collection and transmission processes. One of the most common attacks on WSNs that can target all layers of the protocol stack is the DoS attack. In this study, a unique DoS Intrusion Detection System (DDS) is proposed to detect DoS attacks specific to WSNs. The proposed system is an ensemble intrusion detection system called STLGBM-DDS, which is developed on Apache Spark big data platform in Google Colab environment, combining LightGBM machine learning algorithm, data balancing and feature selection processes. In order to reduce the effects of data imbalance on system performance, data imbalance processing consisting of Synthetic Minority Oversampling Technique (SMOTE) and Tomek-Links sampling methods called STL was used. In addition, Information Gain Ratio was used as a feature selection technique in the data preprocessing stage. The effects of both data balancing and feature selection stages on the detection performance of the system were investigated. The results obtained were evaluated using the Accuracy, F-Measure, Precision, Recall, ROC Curve and Precision-Recall Curve parameters. As a result, the proposed method achieved an overall accuracy of 99.95%. Also, it achieved 99.99%, 99.96%, 99.98%, 99.92%, 99.87% accuracy performance according to Normal, Grayhole, Blackhole, TDMA and Flooding classes, respectively. According to the results obtained, the proposed method has achieved very successful results in DoS attack detection in WSNs compared to current methods.

show abstract

A fast network intrusion detection system using adaptive synthetic oversampling and LightGBM

Cited by 140 publications

References 20 publications

Financial Crisis Prediction in Chinese Real Estate Industry from Cash Flow Perspective Based on Machine Learning

Financial Crisis Prediction in Chinese Real Estate Industry from Cash Flow Perspective Based on Machine Learning

Research on non-destructive testing of hotpot oil quality by fluorescence hyperspectral technology combined with machine learning

STLGBM-DDS: An Efficient Data Balanced DoS Detection System for Wireless Sensor Networks on Big Data Environment

Contact Info

Product

Resources

About