How to Effectively Collect and Process Network Data for Intrusion Detection?

Komisarek, Mikołaj; Pawlicki, Marek; Kozik, Rafał; Hołubowicz, Witold; Choraś, Michał

doi:10.3390/e23111532

Cited by 13 publications

(11 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the prevalent research problems in the domain of intrusion detection is the changing characteristics of both network traffic and the modern threat landscape (Komisarek et al, 2021a). The rate of change in the field is inextricably linked to the intensity of the cyberarms race.…”

Section: Empirical Reviewmentioning

confidence: 99%

Improving Network Security: An Intrusion Detection System (IDS) Dataset from Higher Learning Institutions, Mbeya University of Science and Technology (MUST), Tanzania

Sindika,

Nicholaus,

Hamadi

2024

East Afr. j. inf. technol.

View full text Add to dashboard Cite

Nowadays, Internet-driven culture securing computer networks in Higher Learning Institutions (HLIs) has become a major responsibility. Intrusion Detection Systems (IDS) are crucial for protecting networks from unauthorized activity and cyber threats. This paper examines the process of improving network security by creating a comprehensive IDS dataset using real traffic from HLIs, highlighting the importance of accurate and representative data in improving the system's ability to identify and mitigate future cyber-attacks. The IDS model was created using a variety of machine learning (ML) techniques. Metrics like accuracy, precision, recall, and F1-score were used to assess the performance of each model. The dataset used for training and testing was real-world network traffic data obtained from the institution's computer network. The results showed that the developed IDS obtained exceptional accuracy rates, with Random Forest, Gradient Boosting, and XGBoost models all achieving an accuracy of around 93%. Precision and recall values were likewise quite high across all algorithms. Furthermore, the study discovered that data quality has a substantial impact on IDS performance. Proper data preparation, feature engineering, and noise removal were found to be helpful in improving model accuracy and reducing false positives. While the IDS models performed well throughout validation and testing, implementing such systems in a production setting necessitates careful thought. As a result, the essay also examined the procedures for testing and deploying the IDS models in a real-world scenario. It underlined the significance of ongoing monitoring and maintenance in order to keep the model effective in identifying intrusions. The research aids in the progress of network security in HLI. Educational institutions can better protect their precious assets and sensitive information from cyberattacks by understanding the impact of data quality on IDS performance and implementing effective deployment techniques

show abstract

Section: Empirical Reviewmentioning

confidence: 99%

Improving Network Security: An Intrusion Detection System (IDS) Dataset from Higher Learning Institutions, Mbeya University of Science and Technology (MUST), Tanzania

Sindika,

Nicholaus,

Hamadi

2024

East Afr. j. inf. technol.

View full text Add to dashboard Cite

show abstract

“…Plenty of them focus on the application or bettering of the algorithms, while using outdated NIDS benchmark datasets, or datasets with features which are unobtainable in a real-world setting while working on live traffic. As noted in [45] and [27], current benchmark datasets lack the common feature and label space. Cybersecurity is a domain flourishing with every new component or technology released.…”

Section: Related Workmentioning

confidence: 99%

“…Research on the Machine-Learning (ML) Based Network Intrusion Detection Systems (NIDS) is of high interest in the scientific community and in industrial practice [30][47] [23]. One of the major obstacles to the wide deployment of ML-based NIDS is the necessity to collect labelled data on premises of the protected network [27], since the ML-components need to learn the specific data distributions. The collection of this type of data requires specialist knowledge and adequate resources [35].…”

Section: Introductionmentioning

confidence: 99%

Towards Deployment Shift Inhibition Through Transfer Learning in Network Intrusion Detection

Pawlicki

Kozik

Choraś

2022

Proceedings of the 17th International Conference on Availability, Reliability and Security

View full text Add to dashboard Cite

Currently, machine learning sees growing adoption in numerous domains, including critical applications, like cybersecurity. However, to fully enjoy the benefits of artificial intelligence the end-user has some high barriers to entry to circumnavigate. The deployment of machine-learning-based Network Intrusion Detection Systems requires the collection of labelled data to train the intelligent components. This is an expensive and laborious process, which necessitates expert knowledge in cyberattacks and computer networks. Even when using data collected and labelled on premises, phenomena like concept drift can cause the model to underperform -a concept known as deployment shift. This paper evaluates the use of transfer learning techniques to curb the effects of deployment shift in machine-learning-based network intrusion detection.

show abstract

“…The authors of [ 13 ] establish the minimal amount of data that is sufficient to efficiently train machine learning algorithms in intrusion detection. The authors also identify the most valuable NetFlow-based features that facilitate effective, machine-learning-based network intrusion detection in the real world.…”

mentioning

confidence: 99%

“…Their objectives are reached in a series of experiments with the use of several feature selection techniques, machine learning algorithms and intrusion detection benchmark datasets. The paper [ 13 ] is the result of the EU Horizon 2020 SIMARGL project (simargl.eu).…”

mentioning

confidence: 99%