A Scalable Decomposition Method for the Dynamic Defense of Cyber Networks

Rasouli, Mohammad; Miehling, Erik; Teneketzis, Demosthenis

doi:10.1007/978-3-319-75268-6_4

Cited by 10 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our model in this paper is much more general in that it allows for non-degenerate distributions. Similar to our work, Saghafian (2015) provides an extension of POMDPs, termed Ambiguous POMDPs (APOMDPs), by allowing for ambiguous transition probabilities. However, unlike the RPOMDP framework we study in this paper, the APOMDP approach proposed in Saghafian (2015) uses α-maximin preferences.…”

Section: Literaturementioning

confidence: 95%

“…RPOMDPs with maximin approach have also been studied under the condition that the set of state transition distributions consists of only degenerate distributions; that is, when each distribution refers to one specific transition almost surely (see, e.g., Bernhard (2000), Başar and Bernhard (2008), Bertsekas and Rhodes (1973), Rasouli et al (2018), and Witsenhausen (1966)). Our model in this paper is much more general in that it allows for non-degenerate distributions.…”

Section: Literaturementioning

confidence: 99%

See 1 more Smart Citation

Robust Partially Observable Markov Decision Processes

Rasouli

Saghafian

2018

SSRN Journal

Self Cite

View full text Add to dashboard Cite

In a variety of applications, decisions needs to be made dynamically after receiving imperfect observations about the state of an underlying system. Partially Observable Markov Decision Processes (POMDPs) are widely used in such applications. To use a POMDP, however, a decision-maker must have access to reliable estimations of core state and observation transition probabilities under each possible state and action pair. This is often challenging mainly due to lack of ample data, especially when some actions are not taken frequently enough in practice. This significantly limits the application of POMDPs in real-world settings. In healthcare, for example, medical tests are typically subject to false-positive and false-negative errors, and hence, the decision-maker has imperfect information about the health state of a patient. Furthermore, since some treatment options have not been recommended or explored in the past, data cannot be used to reliably estimate all the required transition probabilities regarding the health state of the patient. We introduce an extension of POMDPs, termed Robust POMDPs (RPOMDPs), which allows dynamic decision-making when there is ambiguity regarding transition probabilities. This extension enables making robust decisions by reducing the reliance on a single probabilistic model of transitions, while still allowing for imperfect state observations. We develop dynamic programming equations for solving RPOMDPs, provide a sufficient statistic and an information state, discuss ways in which their computational complexity can be reduced, and connect them to stochastic zero-sum games with imperfect private monitoring.

show abstract

Section: Literaturementioning

confidence: 95%

Section: Literaturementioning

confidence: 99%

Robust Partially Observable Markov Decision Processes

Rasouli

Saghafian

2018

SSRN Journal

Self Cite

View full text Add to dashboard Cite

show abstract

“…We consider the RHS of ( 44) term-by-term. By direct application of (22) in Definition 4, the first term in the RHS satisfies…”

Section: B Properties Of Approximate Information Statesmentioning

confidence: 99%

“…, T , let m t = (x 0:t , u 0:t−1 ) be the realization of M t and let the approximate information state be xt = µ t (x t ). We first derive the value of t in the RHS of (22) In this appendix, we derive the values of t and δ t for all t = 0, . .…”

Section: Appendix B -Approximation Bounds For Perfectly Observed Systemsmentioning

confidence: 99%

“…At the expense of average performance, this strategy provides concrete guarantees on the worst-case performance during each operation of the system. Thus, this approach has been widely applied to systems under attack from an adversary, e.g., cyber-security [22] or cyber-physical systems [23], and systems where a single failure can be damaging, e.g., water reservoirs [24], or power systems [25].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Approximate Information States for Worst-Case Control and Learning in Uncertain Systems

Dave¹,

Nishanth²,

Malikopoulos³

2023

Preprint

View full text Add to dashboard Cite

In this paper, we investigate discrete-time decisionmaking problems in uncertain systems with partially observed states. We consider a non-stochastic model, where uncontrolled disturbances acting on the system take values in bounded sets with unknown distributions. We present a general framework for decision-making in such problems by developing the notions of information states and approximate information states. In our definition of an information state, we introduce conditions to identify for an uncertain variable sufficient to construct a dynamic program (DP) that computes an optimal strategy. We show that many information states from the literature on worstcase control actions, e.g., the conditional range, are examples of our more general definition. Next, we relax these conditions to define approximate information states using only output variables, which can be learned from output data without knowledge of system dynamics. We use this notion to formulate an approximate DP that yields a strategy with a bounded performance loss. Finally, we illustrate the application of our results in control and reinforcement learning using numerical examples.

show abstract

Adaptation and Deception in Adversarial Cyber Operations

Cybenko

2020

Modeling and Design of Secure Internet of Things

View full text Add to dashboard Cite

A Scalable Decomposition Method for the Dynamic Defense of Cyber Networks

Cited by 10 publications

References 19 publications

Robust Partially Observable Markov Decision Processes

Robust Partially Observable Markov Decision Processes

Approximate Information States for Worst-Case Control and Learning in Uncertain Systems

Adaptation and Deception in Adversarial Cyber Operations

Contact Info

Product

Resources

About