No abstract

Fig. 1: a) Object Memory Transformer (OMT) for efficient indoor navigation to find the target object. Here, the agent looks for "Pillow" on the sofa, which is in the next room and the agent cannot observe it from the start position. Even in this complex case, OMT can take an efficient path to the target shown as a trajectory in the blue dots in (b) by exploiting long-term visual cues while taking into account the relevance of the feature at each time step. Specifically, OMT stores long-term history of observed scenes and objects into Object-Scene Memory (OSM), attends to the crucial ones in OSM, and produces useful features for the RL agent. At the begging of the episode, when the agent is looking at a wall, it rarely attends to those histories (notice at the dark gray region in the attention map of (a)). Once the target comes into the agent's sight, the attention weight gets higher (shown in white).

The Perceptual Crossing Experiment (PCE) has been the object of study for over a decade, and aims at explaining how we perceive, interact with, and understand each other in real-time. In addition to human participant studies, a number of computational models have investigated how virtual agents can solve this task. However, the set of implementation choices that has been explored to date is rather limited, and the large number of variables that can be used make it very difficult to replicate the results. The main objective of this paper is to describe the PCE Simulation Toolkit we have developed and published as an open-source repository on GitHub. We hope that this effort will help make future PCE simulation results reproducible and advance research in the understanding of possible behaviors in this experimental paradigm. At the end of this paper, we present two case studies of evolved agents that demonstrate how parameter choices affect the simulations.