Serial Dependence of Emotion Within and Between Stimulus Sensory Modalities

Abstract

How we perceive the world is not solely determined by what we sense at a given moment in time, but also by what we processed recently. Here we investigated whether such serial dependencies for emotional stimuli transfer from one modality to another. Participants were presented a random sequence of emotional sounds and images and instructed to rate the valence and arousal of each stimulus (Experiment 1). For both ratings, we conducted an intertrial analysis, based on whether the rating on the previous trial was low or high. We found a positive serial dependence for valence and arousal regardless of the stimulus modality on two consecutive trials. In Experiment 2, we examined whether passively perceiving a stimulus is sufficient to induce a serial dependence. In Experiment 2, participants were instructed to rate the stimuli only on active trials and not on passive trials. The participants were informed that the active and passive trials were presented in alternating order, so that they were able to prepare for the task. We conducted an intertrial analysis on active trials, based on whether the rating on the previous passive trial (determined in Experiment 1) was low or high. For both ratings, we again observed positive serial dependencies regardless of the stimulus modality. We conclude that the emotional experience triggered by one stimulus affects the emotional experience for a subsequent stimulus regardless of their sensory modalities, that this occurs in a bottom-up fashion, and that this can be explained by residual activation in the emotional network in the brain.

Serial dependence in emotion perception mirrors the autocorrelations in natural emotion statistics

“…We also observed feature tuning, which shows that participants' ratings for the current trial were influenced by the emotion of the previous trial only if the sequential emotions were similar. This finding supports previous research that has found only similar sequential emotions to be serially dependent ( Palumbo et al, 2017 ; Van der Burg et al, 2021 ). The result is also consistent with other studies that found serial dependence in the affect rating of emotional stimuli ( Palumbo et al, 2017 ; Van der Burg et al, 2021 ).…”

“…This finding supports previous research that has found only similar sequential emotions to be serially dependent ( Palumbo et al, 2017 ; Van der Burg et al, 2021 ). The result is also consistent with other studies that found serial dependence in the affect rating of emotional stimuli ( Palumbo et al, 2017 ; Van der Burg et al, 2021 ). Although many serial dependence studies use low-level stimuli in their investigation ( Kiyonaga et al, 2017 ), serial dependence has been found across many levels beyond just perception, including decision making ( Abrahamyan, Silva, Dakin, Carandini, & Gardner, 2016 ; Braun, Urai, & Donner, 2018 ; Fritsche, Mostert, & de Lange, 2017 ; Lueckmann, Macke, & Nienborg, 2018 ; Pascucci et al, 2019 ) and memory ( Barbosa et al, 2020 ; Kiyonaga et al, 2017 ).…”

“…The results revealed serial dependencies in the perception of emotion in scenes taken from natural movies (including Hollywood films, documentaries, and home videos). These positive serial dependencies suggest that perceived affective content is biased, pulled toward stimuli presented up to 12 seconds or more in the past ( Figure 5 a), consistent with prior research ( Alais, Xu, Wardle, & Taubert, 2021 ; Collins, 2021 ; Fischer & Whitney, 2014 ; Liberman et al, 2018 ; Manassi et al, 2017 ; Manassi et al, 2018 ; Manassi & Whitney, 2022 ; Mei et al, 2019 ; Palumbo, D'Ascenzo, Quercia, & Tommasi, 2017 ; Van der Burg, Toet, A., Brouwer, A.-M., & Van Erp, 2021 ). Echoing previous work, we also found that serial dependence in emotion perception is tuned to the similarity in sequential stimuli—only similar sequential emotions are perceived to be serially dependent.…”

“…It might also change depending on individual differences ( Kondo et al, 2022 ). At first blush, our results might seem at odds with those of Taubert, et al (2016) and consistent with those of Liberman et al (2018) , Mei et al (2019) , Van der Burg et al (2021) , and Collins (2021) . However, our results are, in fact, also consistent with Taubert's suggestion that serial dependence in perception might match the statistics of the world.…”

“…Although serial dependence has been found in affect ratings of visual and auditory stimuli ( Palumbo et al, 2017 ; Van der Burg et al, 2021 ), the temporal tuning of serial dependence in emotion perception has yet to be thoroughly investigated. Additionally, although many previous studies have investigated serial dependence in emotional expressions ( Liberman et al, 2018 ; Liberman & Whitney, 2015 ; Mei et al, 2019 ; Taubert, Alais, et al, 2016 ), no previous study has looked at the serial dependence in emotion ratings of physically unrelated faces embedded in natural context.…”