An amodal shared resource model of language-mediated visual attention

Smith, Alastair Charles; Monaghan, Padraic; Huettig, Falk

doi:10.3389/fpsyg.2013.00528

Cited by 10 publications

(21 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, an alternative conception of the role of semantics is that it interacts with written and spoken forms via a central resource, as shown in Figure 1C (Dilkina, McClelland, & Plaut, 2008;Smith, Monaghan, & Huettig, 2013). Central resource models propose that semantics begins to be activated as quickly as the phonological form of a word and affects its visual identification (both for regular and exception words), echoing neuroimaging findings (e.g., Hauk et al, 2012).…”

Section: Computational Modellingmentioning

confidence: 92%

How Word Meaning Influences Word Reading

Taylor

Duff

Woollams

et al. 2015

Curr Dir Psychol Sci

Self Cite

View full text Add to dashboard Cite

Understanding how we read is a fundamental question for psychology, with critical implications for education. Studies of word reading tend to focus on the mappings between the written and spoken forms of words. In this paper, we review evidence from developmental, neuroimaging, neuropsychological, and computational studies that show that knowledge of word meanings is inextricably involved in word reading. Consequently, models of reading must better specify the role of meaning in skilled reading and its acquisition. Further, our review paves the way for educationally realistic research to confirm whether explicit teaching of oral vocabulary improves word reading.Key words: reading, learning, orthography, phonology, semantics WORD MEANING AND READING / 4 HOW WORD MEANING INFLUENCES WORD READINGReading is fundamental to operating in modern society. For literate adults, reading is immediate, automatic, and efficient. Despite this, reading is not an innate capability but is parasitic upon an earlier acquired spoken language system that includes information about what words sound like (phonology) and what they mean (semantics). The ultimate goal of learning to read is to comprehend texts, for which knowledge of word meanings is essential.The ability to recognise and read aloud words (encompassed by the term word reading throughout this review) is a pre-requisite for reading comprehension. Research has focused on the orthography (print) to phonology connections that underpin word reading. However, the role of word meanings in word reading is less well understood and is thus the focus of this review. As suggested by Balota (1990), seminal findings from the adult behavioural literature now demonstrate that word reading efficiency is significantly influenced by semantic variables, over and above frequency and orthographic factors (Cortese & Schock, 2013). Building on this, we review recent evidence from a number of research traditions, which show conclusively that word reading in alphabetic languages is influenced by knowledge of semantics.Cognitive models of reading all include orthographic, phonological, and semantic representations of words, as well as the mappings between them. In computational models based on a classical dual-route approach ( Figure 1A; Coltheart, Rastle, Perry, Langdon, & Ziegler, 2001;Perry, Ziegler, & Zorzi, 2007), activation of a word's semantics is not necessary for recognising or reading it aloud. In contrast, the triangle modelling tradition ( Figure 1B; Harm & Seidenberg, 2004;Monaghan & Ellis, 2010;Plaut, McClelland, Seidenberg, & Patterson, 1996;Woollams, Lambon Ralph, Plaut, & Patterson, 2007) posits that semantic WORD MEANING AND READING / 5 information directly affects the learning and processing of mappings between the written and spoken forms of words. More specifically, it predicts that meaning particularly aids print-to-sound mapping when this process is difficult (as in words with exceptional spellings, or for poorer readers), and also that skilled reading involves increased...

show abstract

Section: Computational Modellingmentioning

confidence: 92%

How Word Meaning Influences Word Reading

Taylor

Duff

Woollams

et al. 2015

Curr Dir Psychol Sci

Self Cite

View full text Add to dashboard Cite

show abstract

“…To represent such a system, we use the Multimodal Integration Model (MIM) of language processing which integrates concurrent phonological, semantic and visual information in parallel during spoken word processing (Smith, Monaghan, & Huettig, 2013, 2014a, 2014b; see also Monaghan & Nazir, 2009). The model is derived from the Hub-and-Spoke framework (Dilkina et al, 2008(Dilkina et al, , 2010Plaut, 2002;Rogers et al, 2004), a single system architecture that consists of a central resource (hub) that integrates and translates information between multiple modality specific sources (spokes).…”

Section: Models Of Multimodal Integration During Speech Processingmentioning

confidence: 99%

“…It has since been demonstrated that alignment models are also capable of generating rhyme competitor effects if they are exposed to noise in the learning environment, such that onset information is not always a perfect predictor of the target word (Magnuson, Tanenhaus, & Aslin, 2000;Magnuson, Tanenhaus, Aslin, & Dahan, 2003;Smith et al, 2013). Evidence to support such predictions is provided by recent visual world data that demonstrates that onset and rhyme effects on language mediated eye gaze can be modulated by the level of noise participants are exposed to in the speech signal (McQueen & Huettig, 2012).…”

Section: Models Of Multimodal Integration During Speech Processingmentioning

confidence: 99%

The multimodal nature of spoken word processing in the visual world: Testing the predictions of alternative models of multimodal integration

Smith

Monaghan

Huettig

2017

Journal of Memory and Language

Self Cite

View full text Add to dashboard Cite

a b s t r a c tAmbiguity in natural language is ubiquitous, yet spoken communication is effective due to integration of information carried in the speech signal with information available in the surrounding multimodal landscape. Language mediated visual attention requires visual and linguistic information integration and has thus been used to examine properties of the architecture supporting multimodal processing during spoken language comprehension. In this paper we test predictions generated by alternative models of this multimodal system. A model (TRACE) in which multimodal information is combined at the point of the lexical representations of words generated predictions of a stronger effect of phonological rhyme relative to semantic and visual information on gaze behaviour, whereas a model in which sub-lexical information can interact across modalities (MIM) predicted a greater influence of visual and semantic information, compared to phonological rhyme. Two visual world experiments designed to test these predictions offer support for sub-lexical multimodal interaction during online language processing.

show abstract

“…The neural network model used within this paper is based on the ASR model of language mediated eye gaze presented in Smith et al (2013a). The same network architecture (see Fig.…”

Section: Architecturementioning

confidence: 99%

Literacy effects on language and vision: Emergent effects from an amodal shared resource (ASR) computational model

Smith

Monaghan

Huettig

2014

Cognitive Psychology

Self Cite

View full text Add to dashboard Cite

a b s t r a c tLearning to read and write requires an individual to connect additional orthographic representations to pre-existing mappings between phonological and semantic representations of words. Past empirical results suggest that the process of learning to read and write (at least in alphabetic languages) elicits changes in the language processing system, by either increasing the cognitive efficiency of mapping between representations associated with a word, or by changing the granularity of phonological processing of spoken language, or through a combination of both. Behavioural effects of literacy have typically been assessed in offline explicit tasks that have addressed only phonological processing. However, a recent eye tracking study compared high and low literate participants on effects of phonology and semantics in processing measured implicitly using eye movements. High literates' eye movements were more affected by phonological overlap in online speech than low literates, with only subtle differences observed in semantics. We determined whether these effects were due to cognitive efficiency and/or granularity of speech processing in a multimodal model of speech processing -the amodal shared resource model (ASR, Smith, Monaghan, & Huettig, 2013a,b). We found that cognitive efficiency in the model had only a marginal effect on semantic processing and did not affect performance for phonological processing, whereas fine-grained versus coarse-grained

show abstract

An amodal shared resource model of language-mediated visual attention

Cited by 10 publications

References 60 publications

How Word Meaning Influences Word Reading

How Word Meaning Influences Word Reading

The multimodal nature of spoken word processing in the visual world: Testing the predictions of alternative models of multimodal integration

Literacy effects on language and vision: Emergent effects from an amodal shared resource (ASR) computational model

Contact Info

Product

Resources

About