Construction and updating of event models in auditory event processing.

Huff, Markus; Maurer, Annika; Brich, Irina; Pagenkopf, Anne; Wickelmaier, Florian; Papenmeier, Frank

doi:10.1037/xlm0000482

Cited by 17 publications

(14 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to EST, global updating is triggered by an error-detection mechanism – in the sense of a complete reset of the present event model – once the event model does not represent real-world observations any more. Thus, our results add to a growing volume of literature suggesting the existence of incremental updating both on a theoretical (Zwaan, Langston, & Graesser, 1995) and an empirical level (Bailey & Zacks, 2015; Huff et al, 2018; Huff, Meitz, & Papenmeier, 2014; Kurby & Zacks, 2012).…”

Section: Discussionsupporting

confidence: 75%

Filling the gap despite full attention: the role of fast backward inferences for event completion

2019

Self Cite

View full text Add to dashboard Cite

The comprehension of dynamic naturalistic events poses at least two challenges to the cognitive system: filtering relevant information with attention and dealing with information that was missing or missed. With four experiments, we studied the completion of missing information despite full attention. Participants watched short soccer video clips and we informed participants that we removed a critical moment of ball contact in half of the clips. We asked participants to detect whether these moments of ball contact were present or absent. In Experiment 1, participants gave their detection responses either directly during an event or delayed after an event. Although participants directed their full attention toward the critical contact moment, they were more likely to indicate seeing the missing ball contact if it was followed by a causally matching scene than if it was followed by an unrelated scene, both for the immediate and delayed responses. Thus, event completion occurs quickly. In Experiment 2, only a causally matching scene but neither a white mask nor an irrelevant scene caused the completion of missing information. This indicates that the completion of missing information is caused by backward inferences rather than predictive perception. In Experiment 3, we showed that event completion occurs directly during a trial and does not depend on expectations built up after seeing the same causality condition multiple times. In Experiment 4, we linked our findings to event cognition by asking participants to perform a natural segmentation task. We conclude that observers complete missing information during coherent events based on a fast backward inference mechanism even when directing their attention toward the missing information.

show abstract

Section: Discussionsupporting

confidence: 75%

Filling the gap despite full attention: the role of fast backward inferences for event completion

2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our finding of longer encoding times for boundary compared to non-boundary items replicated previous findings in unisensory segmentation (Zacks et al, 2009;Radvansky and Copeland, 2010;Heusser et al, 2018;Huff et al, 2018;van de Ven et al, 2021).…”

Section: Discussionsupporting

confidence: 89%

“…This finding is in agreement with a previous multisensory segmentation study of movie clips (Meitz et al, 2020), in which participants detected between-scene boundaries better than within-scene boundaries regardless of whether the audio track was audible. More generally, our finding may weigh in on the inconsistent results of encoding time of boundary items increasing (Zacks et al, 2009) or not changing with higher dimensional complexity (Experiment 3 in (Huff et al, 2018)). Our finding appears in support of the latter.…”

Section: Discussionmentioning

confidence: 68%

“…Segmentation has often been studied using movie clips, in which the contextual changes are typically multisensory (i.e., audio and visual) (Newtson, 1973;Boltz, 1992;Schwan et al, 2000;Schwan and Garsoffky, 2004;Furman et al, 2007;Zacks et al, 2009;Huff et al, 2014;Baldassano et al, 2017;Chen et al, 2017;Ben-Yakov and Henson, 2018;Cutting, 2019). Segmentation studies that used unisensory contextual features showed comparable segmentation effects in the visual (Zacks et al, 2009;Ezzyat and Davachi, 2011;Newberry and Bailey, 2019) and auditory domain (Sridharan et al, 2007;Baldassano et al, 2018;Huff et al, 2018), suggesting that segmentation is independent of sensory modality. However, the contribution of multisensory integration to temporal segmentation has remained under-investigated.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multisensory integration of contextual boundaries affects temporal order memory, but not encoding or recognition

Ven¹,

Kleuters²,

Stuiver³

2021

Preprint

View full text Add to dashboard Cite

We memorize our daily life experiences, which are often multisensory in nature, by segmenting them into distinct event models, in accordance with perceived contextual or situational changes. However, very little is known about how multisensory integration affects segmentation, as most studies have focused on unisensory (visual or audio) segmentation. In three experiments, we investigated the effect of multisensory integration on segmentation in memory and perception. In Experiment 1, participants encoded lists of visual objects while audio and visual contexts changed synchronously or asynchronously. After each list, we tested recognition and temporal associative memory for pictures that were encoded in the same audio-visual context or that crossed a synchronous or an asynchronous multisensory change. We found no effect of multisensory integration for recognition memory: Synchronous and asynchronous changes similarly impaired recognition for pictures encoded at those changes, compared to pictures encoded further away from those changes. Multisensory integration did affect temporal associative memory, which was worse for pictures encoded at synchronous than at asynchronous changes. Follow up experiments showed that this effect was not due to the higher complexity of multisensory over unisensory contexts (Experiment 2), nor that it was due to the temporal unpredictability of contextual changes inherent to Experiment 1 (Experiment 3). We argue that participants formed situational expectations through multisensory integration, such that synchronous multisensory changes deviated more strongly from those expectations than asynchronous changes. We discuss our findings in light of supportive and conflicting findings of uni- and multisensory segmentation.

show abstract

“…According to Zwaan and Radvansky, such a situation model, which was originally only used to explain word processing, should therefore be accessible to various codalities. Although most research on this eventindexing model and the construction of mental models during narrative comprehension is based on text-based stimulus material (Radvansky & Copeland, 2010), there is converging evidence that comprehenders also construct and update mental models while watching silent movies (Magliano & Zacks, 2011) and audiovisual narratives (Huff, Meitz, & Papenmeier, 2014) or listening to audio dramas (Huff et al, 2018;Papenmeier, Maurer, & Huff, 2019). This suggests that basic processes of narrative understanding are independent of codality (Meitz, Meyerhoff, & Huff, 2019).…”

Section: Representing Dynamic Events In Situation Modelsmentioning

confidence: 99%

Cross-codal integration of bridging-event information in narrative understanding

et al. 2020

Self Cite

View full text Add to dashboard Cite

Visual narratives communicate event sequences by using different code systems such as pictures and texts. Thus, comprehenders must integrate information from different codalities. This study addressed such cross-codal integration processes by investigating how the codality of bridging-event information (i.e., pictures, text) affects the understanding of visual narrative events. In Experiment 1, bridging-event information was either present (as picture or text) or absent (i.e., not shown). The viewing times for the subsequent picture depicting the end state of the action were comparable within the absent and the text conditions. Further, the viewing times for the end-state picture were significantly longer in the text condition as compared to the pictorial condition. In Experiment 2, we tested whether replacing bridging-event information with a blank panel increases viewing times in a way similar to the text condition. Bridging event information was either present (as picture) or absent (not shown vs. blank panel). The results replicated Experiment 1. Additionally, the viewing times for the end-state pictures were longest in the blank condition. In Experiment 3, we investigated the costs related to integrating information from different codalities by directly comparing the text and picture conditions with the blank condition. The results showed that the distortion caused by the blank panel is larger than the distortion caused by cross-codal integration processes. Summarizing, we conclude that cross-codal information processing during narrative comprehension is possible but associated with additional mental effort. We discuss the results with regard to theories of narrative understanding.

show abstract

Construction and updating of event models in auditory event processing.

Cited by 17 publications

References 53 publications

Filling the gap despite full attention: the role of fast backward inferences for event completion

Filling the gap despite full attention: the role of fast backward inferences for event completion

Multisensory integration of contextual boundaries affects temporal order memory, but not encoding or recognition

Cross-codal integration of bridging-event information in narrative understanding

Contact Info

Product

Resources

About