The representational similarity between visual perception and recent perceptual history

Abstract

From moment to moment, the visual properties of objects in the world fluctuate due to external factors like ambient lighting, occlusion and eye movements, and internal (proximal) noise. Despite this variability in the incoming information, our perception is stable. Serial dependence, the behavioral attraction of current perceptual responses towards previously seen stimuli, may reveal a mechanism underlying stability: a spatio-temporally tuned operator that smoothes over spurious fluctuations. The current study examined the neural underpinnings of serial dependence by recording the electroencephalographic (EEG) brain response of female and male human observers to prototypical objects (faces, cars and houses) and morphs that mixed properties of two prototypes. Behavior was biased towards previously seen objects. Representational similarity analysis revealed that responses evoked by visual objects contained information about the previous stimulus. The trace of previous representations in the response to the current object occurred immediately upon object appearance, suggesting that serial dependence arises from a brain state or set that precedes processing of new input. However, the brain response to current visual objects was not representationally similar to the trace they leave on subsequent object representations. These results reveal that while past stimulus history influences current representations, this influence does not imply a shared neural code between the previous trial (memory) and the current trial (perception).Significance statementThe perception of visual objects is pulled towards instances of that object seen in the recent past. The neural underpinnings of this serial dependence remain to be fully investigated. The present study examined EEG responses to faces, cars and houses, and ambiguous between-category morphs. With representational similarity analysis, we showed (1) object-specific neural patterns that differentiate the three categories; (2) that the response to the current object contains information about the previous object, mirroring behavioral serial dependence; (3) that the object-specific neural pattern about the past was different from that in the current response, revealing that while past stimulus history influences current representations, this does not imply a shared neural code between the previous trial (memory) and the current trial (perception).

Reactivated past decisions repel early sensory processing and attract late decision-making

“…PFC recordings on monkeys show that past-trial information is transiently reactivated just before trial onset and further correlates with serial bias behavior 28 . Our recent study 19 as well as previous works [31][32][33][34] demonstrate that past-trial information, indeed retained in activity-silent states during the intertrial interval, is triggered by the corresponding event within the current trial, and the co-occurrence of past and present contributes to serial bias. Most importantly, the past-to-present neural influence exhibits a feature-specific direction commensurate with the corresponding bias behavior, i.e., repulsive for sensory and motor features, and attractive for choices 19 .…”

“…Both repulsive and attractive past-present interactions rely on memory reactivations of past. Consistent with the 'activity-silent' WM theory 49 , recent studies show that past-trial information silently stored in WM could be reactivated either before stimulus onset or after 19,28,[31][32][33][34]50 . Here, only pasttrial information that contributes to serial bias behaviors is reactivated, suggesting a substantial role of memory reactivation in serial dependence.…”

Serial dependence persists longer for imaginary versus real stimuli

“…John-Saaltink et al, 2016). Additionally, when the influence on the current stimulus has been examined, past trial information has often been found to be in an "inverted" code relative to the currently encoded stimulus (Hajonides et al, 2023;J. Luo & Collins, 2023;Papadimitriou et al, 2016;Sheehan & Serences, 2022); see (Ranieri et al, 2022) for a notable exception).…”

Serial dependence in visual perception: A meta-analysis and review

“…Additional debates continue about where serial dependence happens in the brain ( Barbosa et al, 2020 ; Braun, Urai, & Donner, 2018 ; Ceylan et al, 2021 ; de Azevedo Neto & Bartels, 2021 ; Fornaciai, Togoli, & Bueti, 2023 ; John-Saaltink et al, 2016 ; J. Luo & Collins, 2023 ; Sheehan & Serences, 2022 ; van Bergen & Jehee, 2019 ), how many different forms of serial dependence there are, and whether it is just another word for some other history effect like priming ( Galluzzi et al, 2022 ) or central tendency ( Tong & Dubé, 2022 ). Whereas our meta-analysis does not take a position in this respect, we believe all of these debates are clarified and better served with the diagnostic criteria firmly established by the meta-analyses here.…”