Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others

Abstract

To achieve human-like common sense about everyday life, machine learning systems must understand and reason about the goals, preferences, and actions of others. Human infants intuitively achieve such common sense by making inferences about the underlying causes of other agents' actions. Directly informed by research on infant cognition, our benchmark BIB challenges machines to achieve generalizable, common-sense reasoning about other agents like human infants do. As in studies on infant cognition, moreover, we use a violation of expectation paradigm in which machines must predict the plausibility of an agent's behavior given a video sequence, making this benchmark appropriate for direct validation with human infants in future studies. We show that recently proposed, deep-learning-based agency reasoning models fail to show infant-like reasoning, leaving BIB an open challenge.

A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories

“…The experiments are split into 5 tasks: A, B, C, D, E. From A to E, every model is trained on data stipulated purely for each corresponding event category. Following the methodology of existing computational VoE datasets [23,[38][39][40][41], all models are only trained on expected videos. This sums to 375 training scenes, 150 validation scenes and 100 testing scenes for each event category.…”

“…The work in VoE has encouraged recent computational development of models and datasets [23,[38][39][40][41] that challenge artificial agents to independently label possible and impossible scenes in physical events, goal preferences [50] and more. While these datasets mimic real-world VoE experiments, they provide mainly vision data with little to no heuristics that aid learning.…”

Précis ofWhat Babies Know

“…Finally, I look forward to insights from computational cognitive science, leveraging the data from experiments testing large samples of infants recruited through online platforms, large-scale field experiments, or collaborative replications of classic findings. Analyses of data from these sources could serve to evaluate diverse computational models of infant cognition and learning (e.g., Gandhi, Stojnik, Lake, & Dillon, 2021), including the probabilistic generative models discussed in the book. What babies know featured few experiments on infants using any of these methods, leaving rich territory for future books to explore.…”

Modeling dynamic social vision highlights gaps between deep learning and humans

“…Human vision is particularly tuned to process dynamic social scenes from an early age [1], yet social vision remains a substantial open challenge in artificial intelligence (AI) [2]. Current AI models struggle to match even human infants in their ability to understand social scenes [3,4]. Many have argued that incorporating insights from cognitive (neuro)science may improve AI models' performance on social tasks [5,6,7].…”