Meaning and modality: Influences of context, semantic memory organization, and perceptual predictability on picture processing.

Abstract

Words (visual or auditory) and pictures are both physical objects that, through experience, have come to be associated with information not explicitly contained in the physical form of the word or picture itself. In this sense, both pictures and words can be thought of as symbols, or objects that "stand for" information that they do not directly represent. Of course, not all words have clear pictorial counterparts (e.g., function words), and in many cases it would take multiple words to convey the information in a single picture. However, pictures and words can often be used in similar ways and can lead to similar kinds of behavior. In a psychological experiment or in everyday life, for example, seeing either the written word "cat" or a picture of a cat can lead an individual to say /kat/ or to be reminded that cats meow, hate water, chase mice, and so on.A long-standing question in psychology asks how that subset of knowledge that can be conveyed by either a word or a picture is stored and processed. Two general classes of models have been put forward. Multiple semantic system models hypothesize that pictures and words are processed in distinct, specialized semantic systems (e.g., Paivio, 1971Paivio, , 1986Paivio, , 1991Shallice, 1988). In Paivio's "dual-code" model, for example, there is a "logogen" system for word processing and an "imagen" system for picture processing. These systems can communicate with one another but operate independently and have their own organization and pro-

Event-related brain potentials during the monitoring of speech errors

“…pictures, line drawings, or gestures) into a context have reported a more frontal scalp distribution for the N400 effects (e.g. Federmeier, and Kutas, 2001;Ganis, Kutas, and Sereno, 1996;Holle, and Gunter, 2007;Willems et al, 2007). Here, it is suggested that in Experiment 2 the additional attempt to predict upcoming words involves (parts of) the language production system (Pickering, and Garrod, 2007).…”

Picture the difference: electrophysiological investigations of picture processing in the two cerebral hemispheres

“…If this is the case, then we expect to find that the N400 response to pictures presented initially to the right visual field (left hemisphere) will vary with both congruency and category membership, while that to items presented initially to the left visual field (right hemisphere) will vary only with congruency. 1 Alternatively, it may be that the modality differences we have observed between word and picture processing in central vision [22,23] will interact with hemispheric differences, yielding a different pattern of effects in the N400 responses to pictures as a function of visual field of picture presentation than was previously observed for words. We also examine earlier ERP components (N1, P2, and N300) related, respectively, to attentional allocation, visual feature detection, and (possibly) object recognition in order to look for hemispheric differences in the perceptual analysis of pictures and to see how such differences, if observed, relate to later, semantic effects.…”

“…Table 1 gives examples. Sentence-final words in the Federmeier and Kutas studies were replaced in this experiment with black and white line drawings derived from the Snodgrass and Vanderwart set [64] and from a commercial clip art package; these were the same line drawings as were used in Federmeier and Kutas [23]. Each line drawing was normed to ensure naming agreement with the word that it would replace.…”

“…Recently we examined the processing of pictures in central vision, comparing the response to expected exemplars, within category violations, and between category violations (as described previously) in highly constraining and less constraining contexts [23]. At a general level, the response to pictures was functionally similar to that previously observed with the presentation of words in central vision [22]: expected exemplars elicited smaller N400 responses than violations of either type, but within category violations elicited smaller N400s than between category violations.…”

Bases électrophysiologiques de la sémantique des objets

“…Contrairement à l'équipe de Nigam [24], Ganis et ses collègues [25] démontrent que la N400 associée aux images d'objets comporte une latence plus courte et se caractérise par une distribution topographique plus frontale par rapport à celle observée pour les mots. Ces REVUES SYNTHÈSE 195 données, qui plaident en faveur d'un système sémantique multiple, sont confirmées par les récents travaux de Federmeier et Kutas [10,26] qui montrent que la N400 évoquée par les mots prédomine sur les régions centro-pariétales tandis que la N400 et la N300 associées aux images d'objets réels culminent sur les régions frontales. De plus, la N400 évoquée par les images d'objets est plus tardive que la N400 verbale et prédomine sur l'hémisphère gauche contrairement à la N400 verbale.…”