Bayesian Nonparametric Models Characterize Instantaneous Strategies in a Competitive Dynamic Game

Abstract

Previous approaches to investigating strategic social interaction in game theory have predominantly used games with clearly-dened turns and limited choices. However, most real-world social behaviors involve

Naturalistic decision-making: continuous, open-world, and recursive

“…However, in both human and monkey participants, it was found that the key strategic variable was not so much shooters' ability to adjust to their opponents as their ability to advantageously time their final movements on each trial (McDonald et al, 2019;). Such paradigms as this bear some relation to continuous games such as duels and are closely related to pursuit games in differential game theory, but in the latter, focus is typically on computing optimal solutions rather than modeling players' real behavior (Basar and Olsder, 1999;Braun et al, 2009).…”

“…Just as importantly, while control systems are often assumed to be autonomous, with fixed, pre-specified set points, RL can be extended to the case of hierarchical learning and control (Doya et al, 2002;Precup & Sutton, 2000;Pearson & Platt, 2013;Merel et al, 2019;Botvinick et al, 2009). That is, the RL framework can easily accommodate the idea of switching between policies or changing goals within the same policy (McDonald et al, 2019;Iqbal et al, 2019). These high-level changes are most often slower (in the case of goals) or sparser (for policy switches) and map neatly onto the experience of rarer deliberative decisions setting in motion automatic behaviors.…”

“…Two recent paradigms from our groups illustrate precisely this sort of integration of continuous control with sparse strategy changes. In the first, monkeys and humans played a competitive video game against conspecifics that required continuous joystick input (McDonald et al, 2019;Iqbal et al, 2019, Figure 4). Based on the idea of a penalty shot in hockey, one player (the shooter) controlled the motion of a small circle (the puck), while the other (the goalie) controlled the vertical motion of a bar placed at the opposite side of the screen.…”

“…These phenomena can be modeled as the result of a control model applied to an evolving value function incorporating each player's trajectory (Iqbal et al, 2019;McDonald et al, 2020), while directly modeling players' policies allows for the calculation of a measure of instantaneous coupling between the players (McDonald et al, 2019).…”

Human and macaque pairs employ different coordination strategies in a transparent decision game

“…Traditionally, economic games are either played simultaneously (neither agent knows the choice of the other before making its own decision) or sequentially in a certain order; yet real dyadic interactions often play out in a real-time with the partner's actions in direct sight 44,45 . Thus, the timing of one's own and other's actions becomes part of the strategy space 46 . Such a "transparent" continuous time setting can change choice strategies of the agents as compared to the classic simultaneous and sequential settings 38 .…”