Wearable eye-trackers offer exciting advantages over screen-based systems, but their use in research settings has been hindered by significant analytic challenges as well as a lack of published performance measures among competing devices on the market. In this article, we address both of these limitations. We describe (and make freely available) an automated analysis pipeline for mapping gaze data from an egocentric coordinate system (i.e. the wearable eye-tracker) to a fixed reference coordinate system (i.e. a target stimulus in the environment). This pipeline allows researchers to study aggregate viewing behavior on a 2D planar target stimulus without restricting the mobility of participants. We also designed a task to directly compare calibration accuracy and precision across 3 popular models of wearable eye-trackers: Pupil Labs 120Hz Binocular glasses, SMI ETG 2 glasses, and the Tobii Pro Glasses 2. Our task encompassed multiple viewing conditions selected to approximate distances and gaze angles typical for short-to mid-range viewing experiments. This work will promote and facilitate the use of wearable eye-trackers for research in naturalistic viewing experiments.All analyses were completed using R v3.4.0 (R Core Team 2017) , with data formatting using the dplyr package (Wickham et al. 2017) . All linear mixed effects models were performed using the lme4 package (Bates et al. 2015) , and follow-up pairwise comparisons were performed using the lsmeans package (Lenth and Others 2016) . All results plots were created using ggplot2 (Wickham 2009) , ggpubr (Kassambara 2017) , and ggsignif (Ahlmann-Eltze 2017) packages for R . Overall PerformanceThe percentage of removed outlier (>5° from target location) gaze points during preprocessing differed by eye-tracker model. Statistical comparisons revealed the mean percentage of valid gaze points for Pupil Labs (mean: 97.1%; SE: 0.9) was significantly lower than SMI (mean: 98.7%; SE: 0.4) and Tobii (mean: 98.2%; SE: 0.4) (p < 0.001; no significant difference between Tobii and SMI).However, given that all models retained > 97% of all gaze points, this difference had a negligible effect on interpretation of subsequent analyses.We first averaged across all distances and gaze angle conditions, and tested the overall relationships between eye-tracker model and accuracy, and eye-tracker model and precision. AccuracyWe fit a linear mixed effects model to test the relationship between accuracy and eye-tracker. This model included eye-tracker as a fixed effect and subject as a random effect. Eye-tracker was a significant predictor of accuracy ( F (2,78) = 7.44, p < .001) in this model. Follow-up pairwise comparisons between eye-trackers revealed that the Pupil Labs eye-tracker was significantly more accurate than SMI ( t (78) = 2.40, p < .05) and Tobii ( t (78) = 3.81, p < .001). All other comparisons were non-significant at p > .1; see Table 2 , and Fig 4.A .

Sparse rewards are one of the most important challenges in reinforcement learning. In the single-agent setting, these challenges have been addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces. Applying these techniques naively to the multi-agent setting results in individual agents exploring independently, without any coordination among themselves. We argue that learning in cooperative multi-agent settings can be accelerated and improved if agents coordinate with respect to what they have explored. In this paper we propose an approach for learning how to dynamically select between different types of intrinsic rewards which consider not just what an individual agent has explored, but all agents, such that the agents can coordinate their exploration and maximize extrinsic returns. Concretely, we formulate the approach as a hierarchical policy where a high-level controller selects among sets of policies trained on different types of intrinsic rewards and the low-level controllers learn the action policies of all agents under these specific rewards. We demonstrate the effectiveness of the proposed approach in a multi-agent learning domain with sparse rewards.

Understanding the principles by which agents interact with both complex environments and each other is a key goal of decision neuroscience. However, most previous studies have used experimental paradigms in which choices are discrete (and few), play is static, and optimal solutions are known. Yet in natural environments, interactions between agents typically involve continuous action spaces, ongoing dynamics, and no known optimal solution. Here, we seek to bridge this divide by using a “penalty shot” task in which pairs of monkeys competed against each other in a competitive, real-time video game. We modeled monkeys’ strategies as driven by stochastically evolving goals, onscreen positions that served as set points for a control model that produced observed joystick movements. We fit this goal-based dynamical system model using approximate Bayesian inference methods, using neural networks to parameterize players’ goals as a dynamic mixture of Gaussian components. Our model is conceptually simple, constructed of interpretable components, and capable of generating synthetic data that capture the complexity of real player dynamics. We further characterized players’ strategies using the number of change points on each trial. We found that this complexity varied more across sessions than within sessions, and that more complex strategies benefited offensive players but not defensive players. Together, our experimental paradigm and model offer a powerful combination of tools for the study of realistic social dynamics in the laboratory setting.

No abstract

No abstract