Learning Games for Defending Advanced Persistent Threats in Cyber Systems

Zhu, Tianqing; Ye, Dayong; Cheng, Zishuo; Zhou, Wanlei; Yu, Philip S.

doi:10.1109/tsmc.2022.3211866

Cited by 12 publications

(11 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although a growing body of work uses reinforcement learning and game theory to find intrusion response strategies (see Section VII), a direct comparison between the defender strategies learned in our framework and those found in previous work is not feasible for two reasons. First, nearly all of the prior works have developed defender strategies for custom simulations [8], [10], [10], [11], [18]- [27], [33]- [35], [38], [39], [60], [62], [63], [67]- [72], [107]- [119] and there is no obvious way to map their solutions to an emulated environment like ours (see Fig. 1 and Appendix C).…”

Section: A Learning Equilibrium Strategies Through Self-playmentioning

confidence: 99%

“…The game is modeled in different ways depending on the use case. Examples from the literature include: advanced persistent threat games [54], [66]- [68], [71], [108], [109], [150], honeypot placement games [112]- [114], resource allocation games [38], [151], authentication games [8], distributed denial-of-service games [115], [120], situational awareness games [152], [153], moving target defense games [107], [154], and intrusion response games [10], [28], [45], [50], [72], [111], [116]- [119], [121]. These games are formulated using various models from the gametheoretic literature.…”

Section: A Game-theoretic Modeling In Network Securitymentioning

confidence: 99%

“…[7], [118]), Blotto games (see e.g. [38]), differential games (see e.g. [110]), hypergames (see e.g.…”

Section: A Game-theoretic Modeling In Network Securitymentioning

confidence: 99%

“…Third, our method to find effective defender strategies includes using an emulation system in addition to a simulation system. The advantage of our method compared to the simulation-only approaches [10], [11], [18]- [27], [33]- [35], [38], [39], [44]- [46], [50], [52], [52], [64], [69], [70], [72] is that the parameters of our simulation system are determined by measurements from an emulation system instead of being chosen by a human expert. Further, the learned strategies are evaluated in the emulation system, not in the simulation system.…”

Section: Reinforcement Learning For Automated Intrusion Responsementioning

confidence: 99%

“…Third, we provide evaluation results from an emulated infrastructure. This addresses a drawback in related research that relies solely on simulations to learn and evaluate strategies [8], [10], [11], [18]- [27], [33]- [35], [38], [39], [43]- [46], [50], [52], [54], [60], [62]- [72].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers

Hammar¹,

Stadler²

2023

Preprint

View full text Add to dashboard Cite

We study automated intrusion response and formulate the interaction between an attacker and a defender as an optimal stopping game where attack and defense strategies evolve through reinforcement learning and self-play. The gametheoretic modeling enables us to find defender strategies that are effective against a dynamic attacker, i.e. an attacker that adapts its strategy in response to the defender strategy. Further, the optimal stopping formulation allows us to prove that optimal strategies have threshold properties. To obtain nearoptimal defender strategies, we develop Threshold Fictitious Self-Play (T-FP), a fictitious self-play algorithm that learns Nash equilibria through stochastic approximation. We show that T-FP outperforms a state-of-the-art algorithm for our use case. The experimental part of this investigation includes two systems: a simulation system where defender strategies are incrementally learned and an emulation system where statistics are collected that drive simulation runs and where learned strategies are evaluated. We argue that this approach can produce effective defender strategies for a practical IT infrastructure.

show abstract