People’s ability to recognize faces varies to a surprisingly large extent and these differences are hereditary. But cognitive and perceptual processing giving rise to these differences remain poorly understood. Here we compared visual sampling of 10 super-recognizers – individuals that achieve the highest levels of accuracy in face recognition tasks – to typical viewers. Participants were asked to learn, and later recognize, a set of unfamiliar faces while their gaze position was recorded. They viewed faces through ‘spotlight’ apertures varying in size, where the face on the screen was modified in real-time to constrict the visual information displayed to the participant around their gaze position. Higher recognition accuracy in super-recognizers was only observed when at least 36% of the face was visible. We also identified qualitative differences in their visual sampling that can explain their superior recognition accuracy: (1) less systematic focus on the eye region; (2) more fixations to the central region of faces; (3) greater visual exploration of faces in general. These differences were observed in both natural and spotlight viewing conditions, but were most apparent when learning faces and not during recognition. Critically, this suggests that superior recognition performance is founded on enhanced encoding of faces into memory rather than memory retention. Together, our results point to a process whereby super-recognizers construct a more robust memory trace by accumulating samples of complex visual information across successive eye movements.

Accurately recognising faces is fundamental to human social interaction. In recent years it has become clear that people’s accuracy differs markedly depending on viewer’s familiarity with a face and their individual skill, but the cognitive and neural bases of these accuracy differences are not understood. We examined cognitive representations underlying these accuracy differences by measuring similarity ratings to natural facial image variation. Using image averaging, and inspired by the computation of Analysis of Variance, we partitioned image variation into differences between faces (between-identity variation) and differences between photos of the same face (within-identity variation). Contrary to prevailing accounts of human face recognition and perceptual learning, we found that modulation of within-identity variation – rather than between-identity variation – was associated with high accuracy. First, similarity of within-identity variation was compressed for familiar faces relative to unfamiliar faces. Second, viewers that are extremely accurate in face recognition – ‘super-recognisers’ – showed enhanced compression of within-identity variation that was most marked for familiar faces. We also present computational analysis showing that cognitive transformations of between- and within-identity variation make separable contributions to perceptual expertise in unfamiliar and familiar face identification respectively. We conclude that inter- and intra-individual accuracy differences primarily arise from differences in the representation of familiar face image variation.