Modelling face memory reveals task-generalizable representations

Abstract

Current cognitive theories are cast in terms of information processing mechanisms that use mental representations [ 1 – 4 ]. For example, people use their mental representations to identify familiar faces under various conditions of pose, illumination and ageing, or to draw resemblance between family members. Yet, the actual information contents of these representations are rarely characterized, which hinders knowledge of the mechanisms that use them. Here, we modelled the 3D representational contents of 4 faces that were familiar to 14 participants as work colleagues. The representational contents were created by reverse correlating identity information generated on each trial with judgments of the face’s similarity to the individual participant’s memory of this face. In a second study, testing new participants, we demonstrated the validity of the modelled contents using everyday face tasks that generalize identity judgments to new viewpoints, age and sex. Our work highlights that such models of mental representations are critical to understanding generalization behavior and its underlying information processing mechanisms.

Elucidating the Neural Representation and the Processing Dynamics of Face Ensembles

“…Fifth, we visualize and assess mental constructs associated with summary representations by reconstructing their visual appearance from neural data elicited by ensemble perception. Of note, this marks a clear departure from previous uses of image reconstruction aimed at retrieving the appearance of specific stimuli, whether alphanumeric characters (Thirion et al, 2006;Miyawaki et al, 2008), scenes (Naselaris et al, 2009;Nishimoto et al, 2011), or faces (Lee and Kuhl, 2016;Chang and Tsao, 2017;Zhan et al, 2019). In contrast, here we retrieve the visual content of internal representations derived from the structure of ensemble displays as opposed to that of the ensembles themselves.…”

Revealing the information contents of memory within the stimulus information representation framework

“…the relationship 〈Pixel Noise; Face Detection〉) to reverse engineer the information contents of the memory prediction for a face. (b) Memory contents from multivariate identity noise sampling [26]. Individual participants saw six random identities controlled on each trial by a generative model of face information.…”

A model of person perception from voices

“…Studies have shown that neuroimaging approaches may be able to shed light on where and how representations are stored (Tsantani, Kriegeskorte, McGettigan & Garrido, 2019 for voices;Carlin & Kriegeskorte, 2017 for faces) as well as the time courses of representations being processed (Dobs et al, 2019 for faces). Reverse correlation approaches linking stimulus properties to perceptual outcomes have also been successfully applied in the visual domain to probe representations of familiar faces (Zhan, Garrod, van Rijsbergen & Schyns, 2019) -in the auditory domain, similar approaches have been used to describe what representations of smiled speech (Ponsot, Arias & Aucouturier, 2018) and trustworthy or dominant intonation patterns may sound like (Ponsot, Burred, Belin & Aucouturier, 2018; though see Knight, Lavan, Kanber & McGettigan, 2018). Alongside these emerging techniques and methods, we must crucially look further to developing novel task paradigms that go beyond identity-specific recognition and discrimination, allowing us to directly test our claim for a shared mechanism across unfamiliar and familiar voices during person perception.…”