On the mental representations originating during the interaction between language and vision

Mishra, Ramesh Kumar; Marmolejo‐Ramos, Fernando

doi:10.1007/s10339-010-0363-y

Cited by 26 publications

(15 citation statements)

References 90 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Participants may have determined the congruence of actions associated with the feet by attending to the pictures without fixating (Posner 1980) or by using peripheral vision to the lower extremities during directed attention to the head, torso, and hands, and increased looking times to the feet to bring new information to attention (LaBerge and Brown 1989). Indeed, this pattern suggests that participants were directing attention to confirm involvement of the body-effector implied by the verb, which suggests, first, that language may be particularly effective for shifting attention to peripheral and less salient areas (Mishra 2015;Mishra and Marmolejo-Ramos 2010), and, second, that this elicits deeper processing (Bergen et al 2010), possibly related to kinesthetic imagery (Stins et al 2015). That this pattern did not apply similarly to incongruent trials suggests that identifying a mismatch might be accomplished via peripheral attention and shallower processing.…”

Section: Discussionmentioning

confidence: 73%

Attention to body-parts varies with visual preference and verb–effector associations

2017

View full text Add to dashboard Cite

Theories of embodied conceptual meaning suggest fundamental relations between others' actions, language, and our own actions and visual attention processes. Prior studies have found that when people view an image of a neutral body in a scene they first look toward, in order, the head, torso, hands, and legs. Other studies show associations between action verbs and the body-effectors used in performing the action (e.g., "jump" with feet/legs; "talk" with face/head). In the present experiment, the visual attention of participants was recorded with a remote eye-tracking system while they viewed an image of an actor pantomiming an action and heard a concrete action verb. Participants manually responded whether or not the action image was a good example of the verb they heard. The eye-tracking results confirmed that participants looked at the head most, followed by the hands, and the feet least of all; however, visual attention to each of the body-parts also varied as a function of the effector associated with the spoken verb on image/verb congruent trials, particularly for verbs associated with the legs. Overall, these results suggest that language influences some perceptual processes; however, hearing auditory verbs did not alter the previously reported fundamental hierarchical sequence of directed attention, and fixations on specific body-effectors may not be essential for verb comprehension as peripheral visual cues may be sufficient to perform the task.

show abstract

Section: Discussionmentioning

confidence: 73%

Attention to body-parts varies with visual preference and verb–effector associations

2017

View full text Add to dashboard Cite

show abstract

“…Given all the conceptual elements discussed so far it is proposed that narrative text comprehension can be seen as the construction of embodied situation models produced from simulations of the events referred to in a text. The simulations include the products of memory systems and inferential processes and all cognitive processes that are dependent on groups of neuronal structures and activities (Mishra & Marmolejo-Ramos, 2010;Marmolejo-Ramos, 2007a, 2007b.…”

Section: Cognitive Processes Involved In Narrative Text Comprehensionmentioning

confidence: 99%

Text comprehension as a problem solving situation

Marmolejo‐Ramos¹,

Cevasco

2014

Univ Psychol

Self Cite

View full text Add to dashboard Cite

Research in text comprehension has provided details as to how text features and cognitive processes interact in order to build comprehension and generate meaning. However, there is no explicit link between the cognitive processes deployed during text comprehension and their place in higher-order cognition, as in problem solving. The purpose of this paper is to propose a cognitive model in which text comprehension is made analogous to a problem solving situation and that relies on current research on well-known cognitive processes such as inference generation, memory, and simulations. The key characteristic of the model is that it explicitly includes the formulation of questions as a component that boosts representational power. Other characteristics of the model are specified and its extensions to basic and applied research in text comprehension and higher-order cognitive processes are outlined.

show abstract

“…In other words, the likelihood of mapping between language-derived visual representations and vision-derived visual representations is contingent upon the nature of the visual environment. Finally, at least when printed word displays are used, recent results suggest that language–vision mapping can also occur at an orthographic representational level (Salverda and Tanenhaus, 2010; see also Myachykov et al, 2011, for discussion of mapping processes at the syntactic level in a language production task; and Mishra and Marmolejo-Ramos, 2010, for an embodied cognition account).…”

Section: Levels Of Representationmentioning

confidence: 99%

Mechanisms and Representations of Language-Mediated Visual Attention

2012

Self Cite

View full text Add to dashboard Cite

The experimental investigation of language-mediated visual attention is a promising way to study the interaction of the cognitive systems involved in language, vision, attention, and memory. Here we highlight four challenges for a mechanistic account of this oculomotor behavior: the levels of representation at which language-derived and vision-derived representations are integrated; attentional mechanisms; types of memory; and the degree of individual and group differences. Central points in our discussion are (a) the possibility that local microcircuitries involving feedforward and feedback loops instantiate a common representational substrate of linguistic and non-linguistic information and attention; and (b) that an explicit working memory may be central to explaining interactions between language and visual attention. We conclude that a synthesis of further experimental evidence from a variety of fields of inquiry and the testing of distinct, non-student, participant populations will prove to be critical.

show abstract

On the mental representations originating during the interaction between language and vision

Cited by 26 publications

References 90 publications

Attention to body-parts varies with visual preference and verb–effector associations

Attention to body-parts varies with visual preference and verb–effector associations

Text comprehension as a problem solving situation

Mechanisms and Representations of Language-Mediated Visual Attention

Contact Info

Product

Resources

About