Optimal Patrol Planning for Green Security Games with Black-Box Attackers

Xu, Haifeng; Ford, Benjamin J.; Fang, Fei; Dilkina, Bistra; Plumptre, Andrew J.; Tambe, Milind; Driciru, Margaret; Wanyama, Fred; Rwetsiba, Aggrey; Nsubaga, Mustapha; Mabonga, Joshua

doi:10.1007/978-3-319-68711-7_24

Cited by 28 publications

(25 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, Green Security Games (GSG) have been proposed to model the strategic interaction between law enforcement agencies (referred to as defenders) and their opponents (referred to as attackers) in green security domains such as combating poaching, illegal logging and overfishing. Mathematical programming based algorithms are designed to compute the optimal defender strategy, which prescribes strategically randomized patrol routes for the defender (Fang, Stone, and Tambe 2015;Xu et al 2017).…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Green Security Games with Real-Time Information

Wang

Shi

et al. 2019

AAAI

Self Cite

View full text Add to dashboard Cite

Green Security Games (GSGs) have been proposed and applied to optimize patrols conducted by law enforcement agencies in green security domains such as combating poaching, illegal logging and overfishing. However, real-time information such as footprints and agents' subsequent actions upon receiving the information, e.g., rangers following the footprints to chase the poacher, have been neglected in previous work. To fill the gap, we first propose a new game model GSG-I which augments GSGs with sequential movement and the vital element of real-time information. Second, we design a novel deep reinforcement learning-based algorithm, DeDOL, to compute a patrolling strategy that adapts to the real-time information against a best-responding attacker. DeDOL is built upon the double oracle framework and the policy-space response oracle, solving a restricted game and iteratively adding best response strategies to it through training deep Q-networks. Exploring the game structure, DeDOL uses domain-specific heuristic strategies as initial strategies and constructs several local modes for efficient and parallelized training. To our knowledge, this is the first attempt to use Deep Q-Learning for security games.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning for Green Security Games with Real-Time Information

Wang

Shi

et al. 2019

AAAI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Regarding (iii), the results follow from (25) and (27), respectively. Specifically, in (27), since 0 < λ max < 1, the error J(P ) − J trunc,η (P ) goes to 0 exponentially fast as η goes to 0 (N η → ∞).…”

Section: A the Truncated And Conditional Return Time Entropiesmentioning

confidence: 56%

“…The problem of designing robotic surveillance strategies has been widely studied [2], [3], [25], [26]. Stochastic surveillance strategies, which emphasize the unpredictability of the movement of the patroller, are desirable since they are capable of defending against intelligent intruders who aim to avoid detection/capture.…”

Section: Introductionmentioning

confidence: 99%

Markov Chains With Maximum Return Time Entropy for Robotic Surveillance

Duan

George

Bullo

2020

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy is the weighted average, over all graph nodes, of the entropy of the first return times of the Markov chain; this objective function is a function series that does not admit in general a closed form.The paper features theoretical and computational contributions. First, we obtain a discrete-time delayed linear system for the return time probability distribution and establish its convergence properties. We show that the objective function is continuous over a compact set and therefore admits a global maximum; a unique globally-optimal solution is known only for complete graphs with unitary travel times. We then establish upper and lower bounds between the return time entropy and the well-known entropy rate of the Markov chain. To compute the optimal Markov chain numerically, we establish the asymptotic equality between entropy, conditional entropy and truncated entropy, and propose an iteration to compute the gradient of the truncated entropy. Finally, we apply these results to the robotic surveillance problem. Our numerical results show that, for a model of rational intruder over prototypical graph topologies and test cases, the maximum return time entropy chain performs better than several existing Markov chains. arXiv:1803.07705v2 [math.OC]

show abstract

“…This approach captures the spatial evolution over time, i.e., correlates one position at time t to another position at time t + 1. Applications range from robotic patrols [23] to green security games [24], and protection of major infrastructures such as airports [6,25]. Fang et al [26] focuses on protecting mobile targets, which results in a continuous set of strategies for the agents.…”

Section: Security Gamesmentioning

confidence: 99%

An Agent-Based Empirical Game Theory Approach for Airport Security Patrols

2020

View full text Add to dashboard Cite

Airports are attractive targets for terrorists, as they are designed to accommodate and process large amounts of people, resulting in a high concentration of potential victims. A popular method to mitigate the risk of terrorist attacks is through security patrols, but resources are often limited. Game theory is commonly used as a methodology to find optimal patrol routes for security agents such that security risks are minimized. However, game-theoretic models suffer from payoff uncertainty and often rely solely on expert assessment to estimate game payoffs. Experts cannot incorporate all aspects of a terrorist attack in their assessment. For instance, attacker behavior, which contributes to the game payoff rewards, is hard to estimate precisely. To address this shortcoming, we proposed a novel empirical game theory approach in which payoffs are estimated using agent-based modeling. Using this approach, we simulated different attacker and defender strategies in an agent-based model to estimate game-theoretic payoffs, while a security game was used to find optimal security patrols. We performed a case study at a regional airport, and show that the optimal security patrol is non-deterministic and gives special emphasis to high-impact areas, such as the security checkpoint. The found security patrol routes are an improvement over previously found security strategies of the same case study.

show abstract

Optimal Patrol Planning for Green Security Games with Black-Box Attackers

Cited by 28 publications

References 10 publications

Deep Reinforcement Learning for Green Security Games with Real-Time Information

Deep Reinforcement Learning for Green Security Games with Real-Time Information

Markov Chains With Maximum Return Time Entropy for Robotic Surveillance

An Agent-Based Empirical Game Theory Approach for Airport Security Patrols

Contact Info

Product

Resources

About