Reinforcement Learning for Intrusion Detection: More Model Longness and Fewer Updates

Abstract

Several works have used machine learning techniques for network-based intrusion detection over the past few years. While proposed schemes have been able to provide high detection accuracies, they do not adequately handle the changes in network traffic behavior as time passes. Researchers often assume that model updates can be performed periodically as needed, although this is not easily feasible in real-world scenarios. This paper proposes a new intrusion detection model based on a reinforcement learning approach that aims to support extended periods without model updates. The proposal is divided into two strategies. First, it applies machine learning scheme as a reinforcement learning task to long-term learning -maintaining high reliability and high classification accuracies over time. Second, model updates are performed using a transfer learning technique coped with a sliding window mechanism that significantly decreases the need for computational resources and human intervention. Experiments performed using a new dataset spanning 8TB of data and four years of real network traffic indicate that current approaches in the literature cannot handle the evolving behavior of network traffic. Nevertheless, the proposed technique without periodic model updates achieves similar accuracy rates to traditional detection schemes implemented with semestral updates. In the case of performing periodic updates on our proposed model, it decreases the false positives up to 8%, false negatives up to 34%, with an accuracy variation up to only 6%, while demanding only seven days of training data and almost five times fewer computational resources when compared to traditional approaches.

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers

“…A large number of studies have focused on applying reinforcement learning to use cases similar to the intrusion response use case we discuss in this paper [9]- [11], [17]- [52], [64], [72]. These works use a variety of models, including MDPs [20], [23], [25], [26], [31], [34], [36], [42], [51], [52], [64], Stochastic games [10], [18], [28], [33], [45], [72], attack graphs [35], Petri nets [43], and POMDPs [9], [11], [21], [27], as well as various reinforcement learning algorithms, including Q-learning [18], [20], [23], [40], [43], [48], [64], [69], SARSA [21], PPO [10], [11], [34], [35], [37], hierarchical reinforcement learning [25], DQN [26], [36]-…”

“…This novel formulation allows us a) to derive and prove structural properties of optimal strategies; and b) to find defender strategies that are effective against an attacker with a dynamic strategy. We thus address a key limitation of many related works, which only consider static attackers [9], [11], [17], [20], [21], [23], [25]- [27], [31], [36], [37], [39], [40], [42], [48], [51]- [53], [60]- [65]. Second, we propose T-FP, an efficient reinforcement learning algorithm that exploits threshold properties of optimal stopping strategies and outperforms a state-of-the-art algorithm for our use case.…”

Enhanced Intrusion Detection with Data Stream Classification and Concept Drift Guided by the Incremental Learning Genetic Programming Combiner

“…In [ 27 ], a novel intrusion detection model based on reinforcement learning was proposed. The model is designed to operate for extended periods without frequent updates and consists of two strategies.…”

Artificial intelligence enabled Luong Attention and Hosmer Lemeshow Regression Window‐based attack detection in 6G

“…An ensemble‐based intrusion detection system was developed in Abbas et al 29 to consider both on binary and multi‐class classification scenarios. A new intrusion recognition mechanism based on reinforcement learning was designed in dos Santos et al 30 with a minimum false positive rate. But, the time was not focused.…”