Solving zero-sum one-sided partially observable stochastic games

Horák, Karel; Bošanský, Branislav; Kovařík, Vojtěch; Kiekintveld, Christopher

doi:10.1016/j.artint.2022.103838

Cited by 6 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The right plot shows that HSVI reaches an HSVI approximation error below 5 within an hour of processing time. Based on the recent literature we anticipated a much longer processing time [82], [122]. This suggests to us that T-FP and HSVI have similar convergence properties.…”

Section: B Evaluating the Learned Strategiesmentioning

confidence: 82%

“…, [82]. Further, from Markov decision theory we know that for any strategy pair (π 1 , π 2 ), a corresponding pair of pure best response strategies (π 1 , π2 ) ∈ B 1 (π 2 ) × B 2 (π 1 ) exists [83].…”

Section: A Analyzing Best Responses Using Optimal Stopping Theorymentioning

confidence: 99%

“…Based on [9], [82], [86]- [88], we formulate Theorem 1, which contains an existence result for equilibria and a structural result for best response strategies of the game.…”

Section: A Analyzing Best Responses Using Optimal Stopping Theorymentioning

confidence: 99%

“…Computing Nash equilibria for a POSG is generally intractable [82]. However, approximate solutions can be obtained through iterative methods.…”

Section: B Finding Nash Equilibria Through Fictitious Self-playmentioning

confidence: 99%

“…Baseline algorithms. We compare the performance of T-FP with that of two popular algorithms in previous work that use reinforcement learning and study use cases similar to ours [70], [82], [103]- [105]. The first algorithm is Neural Fictitious Self-Play (NFSP) [106], which is a general fictitious selfplay algorithm that does not exploit the threshold structures expressed in Theorem 1.…”

Section: A Learning Equilibrium Strategies Through Self-playmentioning

confidence: 99%

See 4 more Smart Citations

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers

Hammar¹,

Stadler²

2023

Preprint

View full text Add to dashboard Cite

We study automated intrusion response and formulate the interaction between an attacker and a defender as an optimal stopping game where attack and defense strategies evolve through reinforcement learning and self-play. The gametheoretic modeling enables us to find defender strategies that are effective against a dynamic attacker, i.e. an attacker that adapts its strategy in response to the defender strategy. Further, the optimal stopping formulation allows us to prove that optimal strategies have threshold properties. To obtain nearoptimal defender strategies, we develop Threshold Fictitious Self-Play (T-FP), a fictitious self-play algorithm that learns Nash equilibria through stochastic approximation. We show that T-FP outperforms a state-of-the-art algorithm for our use case. The experimental part of this investigation includes two systems: a simulation system where defender strategies are incrementally learned and an emulation system where statistics are collected that drive simulation runs and where learned strategies are evaluated. We argue that this approach can produce effective defender strategies for a practical IT infrastructure.

show abstract

Section: B Evaluating the Learned Strategiesmentioning

confidence: 82%

Section: A Analyzing Best Responses Using Optimal Stopping Theorymentioning

confidence: 99%

“…Based on [9], [82], [86]- [88], we formulate Theorem 1, which contains an existence result for equilibria and a structural result for best response strategies of the game.…”

Section: A Analyzing Best Responses Using Optimal Stopping Theorymentioning

confidence: 99%

“…Computing Nash equilibria for a POSG is generally intractable [82]. However, approximate solutions can be obtained through iterative methods.…”

Section: B Finding Nash Equilibria Through Fictitious Self-playmentioning

confidence: 99%

Section: A Learning Equilibrium Strategies Through Self-playmentioning

confidence: 99%

See 3 more Smart Citations

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers

Hammar¹,

Stadler²

2023

Preprint

View full text Add to dashboard Cite

show abstract

Partially Observable Stochastic Games with Neural Perception Mechanisms

Yan,

Santos,

Norman

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Stochastic games are a well established model for multi-agent sequential decision making under uncertainty. In practical applications, though, agents often have only partial observability of their environment. Furthermore, agents increasingly perceive their environment using data-driven approaches such as neural networks trained on continuous data. We propose the model of neuro-symbolic partially-observable stochastic games (NS-POSGs), a variant of continuous-space concurrent stochastic games that explicitly incorporates neural perception mechanisms. We focus on a one-sided setting with a partially-informed agent using discrete, data-driven observations and another, fully-informed agent. We present a new method, called one-sided NS-HSVI, for approximate solution of one-sided NS-POSGs, which exploits the piecewise constant structure of the model. Using neural network pre-image analysis to construct finite polyhedral representations and particle-based representations for beliefs, we implement our approach and illustrate its practical applicability to the analysis of pedestrian-vehicle and pursuit-evasion scenarios.

show abstract

Intrusion Tolerance as a Two-Level Game

Hammar,

Stadler

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Solving zero-sum one-sided partially observable stochastic games

Cited by 6 publications

References 33 publications

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers

Learning Near-Optimal Intrusion Responses Against Dynamic Attackers

Partially Observable Stochastic Games with Neural Perception Mechanisms

Intrusion Tolerance as a Two-Level Game

Contact Info

Product

Resources

About