A Decentralized Partially Observable Markov Decision Model with Action Duration for Goal Recognition in Real Time Strategy Games

Abstract

Multiagent goal recognition is a tough yet important problem in many real time strategy games or simulation systems. Traditional modeling methods either are in great demand of detailed agents’ domain knowledge and training dataset for policy estimation or lack clear definition of action duration. To solve the above problems, we propose a novel Dec-POMDM-T model, combining the classic Dec-POMDP, an observation model for recognizer, joint goal with its termination indicator, and time duration variables for actions with action termination variables. In this paper, a model-free algorithm named cooperative colearning based on Sarsa is used. Considering that Dec-POMDM-T usually encounters multiagent goal recognition problems with different sorts of noises, partially missing data, and unknown action durations, the paper exploits the SIS PF with resampling for inference under the dynamic Bayesian network structure of Dec-POMDM-T. In experiments, a modified predator-prey scenario is adopted to study multiagent joint goal recognition problem, which is the recognition of the joint target shared among cooperative predators. Experiment results show that (a) Dec-POMDM-T works effectively in multiagent goal recognition and adapts well to dynamic changing goals within agent group; (b) Dec-POMDM-T outperforms traditional Dec-MDP-based methods in terms of precision, recall, andF-measure.

Answerable and Unanswerable Questions in Risk Analysis with Open‐World Novelty

“…Algorithms developed for inference and learning in probabilistic graphical models such as DBNs can be modified and applied to optimize polices for SMDPs (Hoffman et al, 2012;Yin et al, 2016). Multiagent planning (MAP) and coordination tasks (e.g., multirobot cooperation in robot soccer or in search-andrescue operations, or in engagements in military and security applications) typically require asynchronous decisions by the agents, as well as frequent updating of inferences about each other's goals and plans, and hence revisions in each agent's own goals and plans (Jiao, Xu, Yue SWei, & Sun, 2017). They are thus naturally modeled by combining the partial observability of POMDPs with the random timing of SMDPs.…”

Performance Analysis of Switched Control Systems Under Common‐source Digital Upsets Modeled by MDHMM

“…MDHMM, a signal statistical analysis model, is widely used in the fields of pattern recognition and financial areas, and many researches focus on the applications of movement segments for controlling a multifunctional prosthetic hand, measurement assessment and face recognition, etc. [15][16][17][18]. Reference [19] proposed a zero-delay MDHMM in terms of the fitting capacity and prediction power to capture the evolution of the foreign exchange rate data under different frequent trading environments.…”