Inspecting a picture before reading affects attentional processing but not comprehension

Abstract

This study investigated whether presenting a picture before reading can encourage situation-model construction. We compared two conditions (n ¼ 30) which differed in whether a picture of the initial situation described in a narrative text was presented before reading (i.e. pictorial-support condition) or not (i.e. no-picture condition). Situation-model construction was measured using both process-and product-oriented measures. Eye-tracking data indicated online resource allocation to the different levels of text representation: surface, textbase, and situation model. Literal text questions and inference questions were used as an offline indication of textbase and situation-model processing, respectively. The results showed that a picture presented before reading led to a redistribution of processing resources during reading, evidenced by a shift from textbase to situation-model processing. This attentional shift did not translate into higher comprehension scores. The results were interpreted in line with multimedia learning theories suggesting pictures can serve as a mental scaffold for situation-model construction.

Does the Timing of Visual Support Affect Sentence Comprehension? An Eye-Tracking Study

“…The authors reported that the eye-movement patterns of the participants who visualized the supporting picture reflected increased effort in processing information to generate inferences, whereas those who did not visualize the picture were more focused on processing literal information. Although there were no differences in text comprehension outcomes between both conditions (for tentative explanations see Wassenburg et al, 2020), this result indicates that visualizing a supporting image shapes subsequent reading behaviour.…”

“…To fill this gap, Eitel and Scheiter called for increasing research analysing readers' eye movements during the reading course. Two exceptions to the lack of this type of research are the studies by Ferreira et al (2013) and, more recently, by Wassenburg et al (2020). Ferreira et al (2013) established that a preview of objects is critical to the observance of garden-path effects in the visual world paradigm.…”

“…Importantly, the study by Ferreira et al (2013) did not explore the processing of texts but of auditory information. In the study by Wassenburg et al (2020), the authors examined whether a picture presented before reading facilitates comprehension of a 404-word narrative text by fostering situation model construction, as compared to the same text without the picture. The authors reported that the eye-movement patterns of the participants who visualized the supporting picture reflected increased effort in processing information to generate inferences, whereas those who did not visualize the picture were more focused on processing literal information.…”

“…Thus, supporting images seem to positively affect text processing. However, both in the study by Wassenburg et al (2020) and in those by Rivero-Contreras et al (2021, 2023, the presentation timing of the pictures was not manipulated, as they were presented only prior or simultaneously to the text, respectively. To fill this gap in research and provide cumulative evidence on online text processing, we conducted a study in which we experimentally compared the effect of presenting supporting images prior vs concurrent with the text on readers' eye movements.…”

“…In the present study, we aimed to build on the lines of research presented above (Eitel et al, 2013;Rivero-Contreras et al, 2021, 2023; Wassenburg et al, 2020) We manipulated word frequency (high vs. low) and visual support (image vs. no image) in a sentence-reading task, which involved eye tracking.…”