Seeing what they read and hearing what they say: Readers' representation of the story characters' world

Abstract

Do readers "see" the words that story characters read and "hear" the words that they hear? Just as priming effects are reduced when stimuli are presented cross-modally on two different occasions, we found reduced transfer effects when story characters were described as experiencing stimuli cross-modally. In Experiment 1, a repeated phrase was described as being part of a spoken message in both Story A and Story B, and transfer effects were found. In Experiment 2, in contrast, when the phrase was described as a written note in one story and a spoken message in the other, reading-time results indicated that readers did not retrieve the meaning of the repeated phrase. The results are consistent with findings indicating that visual imagery simulates visual processing and that auditory imagery simulates auditory processing. We conclude that readers mentally simulate the perceptual details involved in story characters' linguistic exchanges.

When story characters communicate: readers’ representations of characters’ linguistic exchanges

“…In Experiment 1, the materials were very similar to those used by Klin and Drumm (2010). In story A, the message in the repetition line was described as having been spoken, and in story B, it was described as being part of a note.…”

“…Because the findings of Klin and Drumm (2010) were surprising, and because they have potentially important implications for our understanding of the way in which readers understand linguistic exchanges, in the present article, we explore these findings further. More specifically, we ask what aspects of the mismatch in the phrases were responsible for the elimination of the transfer effects.…”

“…However, when the phrase was part of a note in story A and part of a conversation in story B, transfer effects were eliminated (Klin & Drumm, 2010). Klin and Drumm concluded that readers encoded the perceptual features of story characters' linguistic exchanges and that the mismatch (visual vs. auditory) eliminated transfer effects.…”

“…However, in contrast to the attention paid to understanding how readers represent descriptions of a character's motor movements, location, or goals, little is known about how readers represent characters' linguistic exchanges. Klin and Drumm (2010) hypothesized that readers mentally simulate a character's linguistic experiences much as they mentally simulate a character's motor actions. This hypothesis is consistent with evidence from brain-imaging studies in which the processing of phrases describing visual, tactile, auditory, gustatory, olfactory, kinesthetic, and somatic perception (e.g., "to hear a shot," "to see a candle"; Belardinelli et al, 2009;Palmiero et al, 2009) led to significant activation of brain regions that are generally associated with the use of those senses.…”

“…Even in this condition, when readers processed an ambiguous phrase that had previously been encountered two passages earlier, not only did they retrieve the phrase from memory, but also they retrieved their sarcastic interpretation of it. Klin and Drumm (2010) went beyond a demonstration of context-independent repetition effects and used this paradigm to investigate the nature of the readers' representation of the repeated phrase. More specifically, they asked whether the readers would encode the perceptual details of the story characters' linguistic exchanges.…”