Processing of written and spoken words: Evidence for common coding

Abstract

Common processing systems involved during reading and listening were investigated. Semantic, phonological, and physical systems were examined using an experimental procedure that involved simultaneous presentation of two words: one visual and one auditory. Subjects were instructed to attend to only one modality and to make responses on the basis of words presented in that modality. Influence of unattended words on semantic and phonological decisions indicated that these processing systems are common to the two modalities. Decisions in the physical task were based on modality-specific codes operating prior to the convergence of information from the two modalities.Studies concerned with word processing have tended to approach the topic by investigating either reading or listening, exclusively, largely ignoring the question of common processing systems for the two tasks. Yet, at some level of stimulus analysis, printed words and spoken words must share processing. The present research examines the relationship between the internal codes involved during reading and during listening and investigates at what level! of stimulus analysis it might be possible to merge the research from these two areas.Evidence from bisensory tasks bears upon the question of common semantic coding of visually and auditorily presented words. Lewis (1972) and Sen and Posner (1979) have found facilitation in pronunciation latency for attended words in either modality when the same word is simultaneously presented on the unattended modality. Similarly, when an unattended auditory digit is the same as the attended visual digit, pronunciation latency for the digit is facilitated (Greenwald, 1970;Mynatt, 1977). It has also been shown that unattended visual (Lewis, 1972) and auditory (Greenwald, 1970;Mynatt, 1977) words and digits interfere with pronunciation of items presented to the attended modality when the attended and unattended items are semantically related. This influence of semantically related unattended words indicates automatic activation of a semantic code that is shared by the two modalities. This paper is based on a dissertation submitted to the graduate school of the University of Oregon in partial fulfillment of degree requirements. I am grateful to Michael Posner for his guidance throughout this project. I also wish to thank Ursula Bellugi and Eleanor Gibson for their helpful comments on earlier drafts of the manuscript and Fred Chang for his assistance with data analysis. The investigation and writing of this report were supported by National Institute of General Medical Sciences Training Grant 5 TOI GM 02165 BHSand by National Institutes of Health National Research Service Award I F32 NS06l09-{)1 from the Division of Neurosciences and Communicative Disorders and Stroke. Requests for reprints should be sent to Haskins Laboratories, 270 Crown Street, New Haven, CT 06510.Copyright 1981 Psychonomic Society, Inc. 93During reading and listening tasks, is there similarly automatic activation of a shared phonological code? For the pr...

Multimodal versus Unimodal Information Processing of Words

“…Investigators have been concerneZd with determining conditions under which facilitation or inhibition occur. Hanson (14) found facilitation when words in the visual and auditory modalities were semantically related and/or redundant; however, in her study attention was directed to one modality at a time.…”

Bimodal Information Processing in a Sonar Task

“…This "redundant signals" effect has been particularly evident in studies with a focus on signal detection sensitivity and/or accuracy. Some of these studies have found superior detection performance in a dual-modality presentation as compared to either single modality presentation (2,6,(9)(10).…”

“…Studies which have compared single and dual-modality presentation conditions on sonar-like tasks have yielded mixed findings (1,(10)(11)(12)(13), although most investigators have concluded that a combined visual and auditory approach should be retained in sonar operation. Recently, Lewandowski and Kobus (14) found that detection threshold was lowered by more than 1.2dB to both visual and auditory targets when they were presented simultaneously rather than separately.…”

Use of orthographic structure by deaf adults: Recognition of fingerspelled words

“…For hearing persons, knowledge of written and spoken words are interrelated: Reading involves relating written words to spoken words (Baddeley, 1979;Hanson, 1981;Liberman, Liberman, Mattingly, & Shankweiler, 1980), and there is evidence that phoneme analysis of spoken words is influenced by access to the written representation of the words (Ehri & Wilce, 1980;Morais, Cary, Alegria, & Bertelson, 1979;Seidenberg & Tanenhaus, 1979). The underlying regularities governing orthographic structure are thought to be abstracted through this experience in relating the written and spoken systems (Ehri, 1980;Gibson, Shurcliff, & Yonas, 1970;Liberman et al, 1980).…”