Decision assistance agent in real-time simulation

Khouj, Mohammed; Sarkaria, Sarbjit; Martí, José

doi:10.1504/ijcis.2014.062968

Cited by 4 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have chosen to employ Monte Carlo estimation because it is well suited to learning from episodic problems of the type encountered in the disaster mitigation domain. Experimental results involving similar work [8] reveal that convergence using step-by-step updates as prescribed by temporal difference learning take 2.6 times longer than episode-by-episode based updates as used in Monte Carlo estimation. The goal of the learning agent is to approximate the optimal action-value function leading to the best long-term reward that corresponds to the best trajectory.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…The scenarios are modeled using i2Sim, a hybrid discrete-time simulator, which can handle vast numbers of interactions with the reinforcement learning agent. The simulated environment is based on an urban community similar to the Downtown Vancouver Model [8]. The model incorporates four electrical power substations (P1, P2, P3 and P4), a water pumping station (W) and infrastructure assets such as venues (V1 and V2) and hospitals (H1 and H2).…”

Section: Intelligent Decision Makingmentioning

confidence: 99%

“…The choice of Monte Carlo estimation over the temporal difference and dynamic programming approaches is also motivated by the need to reduce intrasystem communications in the architecture. In a reinforcement learning with temporal difference (RL-TD) approach, communications between the agent and i2Sim (Figure 4) introduce an overhead that is incurred at every time step [8]. In fact, a significant portion of the computational time is due to the MATLAB-Java communications interface alone.…”

Section: Rl-mc Based Learningmentioning

confidence: 99%

See 2 more Smart Citations

Reinforcement Learning Using Monte Carlo Policy Estimation for Disaster Mitigation

Khouj

Sarkaria

López

et al. 2014

Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications

Self Cite

View full text Add to dashboard Cite

Part 3: Infrastructure Modeling and SimulationInternational audienceUrban communities rely heavily on the system of interconnected critical infrastructures. The interdependencies in these complex systems give rise to vulnerabilities that must be considered in disaster mitigation planning. Only then will it be possible to address and mitigate major critical infrastructure disruptions in a timely manner.This paper describes an intelligent decision making system that optimizes the allocation of resources following an infrastructure disruption. The novelty of the approach arises from the application of Monte Carlo estimation for policy evaluation in reinforcement learning to draw on experiential knowledge gained from a massive number of simulations. This method enables a learning agent to explore and exploit the available trajectories, which lead to an optimum goal in a reasonable amount of time. The specific goal of the case study described in this paper is to maximize the number of patients discharged from two hospitals in the aftermath of an infrastructure disruption by intelligently utilizing the available resources. The results demonstrate that a learning agent, through interactions with an environment of simulated catastrophic scenarios, is capable of making informed decisions in a timely manner

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

Section: Intelligent Decision Makingmentioning

confidence: 99%

Section: Rl-mc Based Learningmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning Using Monte Carlo Policy Estimation for Disaster Mitigation

Khouj

Sarkaria

López

et al. 2014

Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…The combinatorial solution of i2Sim uses the discrete HRT tables to find the optimum combination of rows across all cells in the system that maximizes the output objective function over a certain time scenario. Two optimization methods that have been successfully applied include reinforcement learning [14] and ordinal optimization [7].…”

Section: I2simmentioning

confidence: 99%

“…This version can achieve orders of magnitude faster solutions and can be used as a good first-order approximation to many problems or as a starting base-point for systems with stronger nonlinearities. The optimization along a time line of the event can be obtained using machine learning techniques such as reinforcement learning [14].…”

Section: I2simmentioning

confidence: 99%

Phenomenological Simulators of Critical Infrastructures

Tofani

D’Agostino

Martí

2016

Managing the Complexity of Critical Infrastructures

View full text Add to dashboard Cite

The objective of this chapter is to introduce and discuss the main phenomenological approaches that have been used within the CI M&S area. Phenomenological models are used to analyse the organizational phenomena of the society considering its complexity (finance, mobility, health) and the interactions among its different components. Within CI MA&S, different modelling approaches have been proposed and used as, for example, physical simulators (e.g. power flow simulators for electrical networks). Physical simulators are used to predict the behaviour of the physical system (the technological network) under different conditions. As an example, electrical engineers use different kind of simulators during planning and managing of network activities for different purposes: (1) power flow simulators for the evaluation of electrical network configuration changes (that can be both deliberate changes or results from of the effects of accidents and/or attacks) and contingency analysis, (2) real time simulators for the design of protection devices and new controllers. For the telecommunication domain one mat resort to network traffic simulators as for example ns2/ns3 codes that allow the simulation of telecommunication networks (wired/wireless) at packet switching level and evaluate its performances. Single domains simulators can be federated to analyse the interactions among different domains. In contrast, phenomenological simulators use more abstract data and models for the interaction among the different components of the system. The chapter will describe the main characteristic of some of the main simulation approaches resulting from the ENEA and UBC efforts in the CIP and Complexity Science field.

show abstract