Simulating the brain-body-environment trinity in closed loop is an attractive proposal to investigate how perception, motor activity and interactions with the environment shape brain activity, and vice versa. The relevance of this embodied approach, however, hinges entirely on the modeled complexity of the various simulated phenomena. In this article, we introduce a software framework that is capable of simulating large-scale, biologically realistic networks of spiking neurons embodied in a biomechanically accurate musculoskeletal system that interacts with a physically realistic virtual environment. We deploy this framework on the high performance computing resources of the EBRAINS research infrastructure and we investigate the scaling performance by distributing computation across an increasing number of interconnected compute nodes. Our architecture is based on requested compute nodes as well as persistent virtual machines; this provides a high-performance simulation environment that is accessible to multi-domain users without expert knowledge, with a view to enable users to instantiate and control simulations at custom scale via a web-based graphical user interface. Our simulation environment, entirely open source, is based on the Neurorobotics Platform developed in the context of the Human Brain Project, and the NEST simulator. We characterize the capabilities of our parallelized architecture for large-scale embodied brain simulations through two benchmark experiments, by investigating the effects of scaling compute resources on performance defined in terms of experiment runtime, brain instantiation and simulation time. The first benchmark is based on a large-scale balanced network, while the second one is a multi-region embodied brain simulation consisting of more than a million neurons and a billion synapses. Both benchmarks clearly show how scaling compute resources improves the aforementioned performance metrics in a near-linear fashion. The second benchmark in particular is indicative of both the potential and limitations of a highly distributed simulation in terms of a trade-off between computation speed and resource cost. Our simulation architecture is being prepared to be accessible for everyone as an EBRAINS service, thereby offering a community-wide tool with a unique workflow that should provide momentum to the investigation of closed-loop embodiment within the computational neuroscience community.

Human-robot interaction is an important component for robots operating in human environments and verbal interaction is in many cases the most intuitive and effective solution for humans. Managing dialogues between physical agents interacting in a physical environment brings additional challenges to virtual dialogue systems (eg. Siri or Google Now). More channels of information are available, as gaze or hands movements, which can modify or support verbal information. Also exophoric references to different parts of the environment can occur along the conversation. In this article we focus on the problem of extending the representation of the dialogue context to a physical environment and using this representation for resolving exophoric references. We also describe an architecture for integrating an open source dialogue manager in a service robot. In this architecture, the aforementioned representation is jointly built by different modules and it's used by the dialogue manager to ground utterances happening in the conversation. Finally we describe several experiments performed for assessing the utility of this architecture with actual robots in physical scenarios.

No abstract