Multimedia design: the effects of relating multimodal information

Abstract

Few models describe learner behaviour during the simultaneous processing of several types of information, yet this is the defining characteristic of the use of multimedia tools, which bring together media in different informational formats (fixed or moving images, sound, text). Following studies in cognitive psychology concerning the increase in the ability to form mental images of words, this study was aimed at defining how different multimedia presentation modes affect the learning of foreign language vocabulary (Russian). A statistically significant effect was observed on word memorisation in the different information presentation modes, suggesting better processing when there is co‐referencing of the different sources, especially when the encoding and tests modes are the same. In addition to these experimental results, some principles for the design of multimodal learning tools are discussed.

Evaluating types and combinations of multimodal presentations in the retention and transfer of concrete vocabulary in EFL learning

“…Their theoretical rationale for this claim was that "when textual, visual, and auditory materials are integrated in this way, the learner may be forced to engage in additional processing that leads to better memorisation" (Dubois & Vial, 2000: 159). Among the results obtained, Dubois and Vial (2000) explain that auditory information presented together with visual elements fostered more learning than textual information presented with the same image, thus confirming the findings of other authors like Mayer and Moreno (1998). Dubois and Vial (2000: 163) explain the consistency of the results saying that "a presentation where both elements presented (image and text) are only visual results in less learning than if both are visual and auditory (less cognitive load)".…”

“…Vocabulary acquisition became the focus of attention in theories of language learning (McCarthy, 1990;Lewis, 1993;Coady & Huckin, 1997;Read, 2000;Schmitt, 2000;Nation, 2001) and in terms of the vocabulary learning strategies students needed to use to attain an effective pool of language for communicative purposes (Gairns & Redman, 1986;Carter & McCarthy, 1988;Schmitt, 1997;Sökmen, 1997). Vocabulary acquisition is here reviewed in studies dealing with second language multimodal learning (Plass, Chun, Mayer & Leutner, 1998;Dubois & Vial, 2000;Farías, Obilinovic & Orrego, 2009;Syrodenko, 2010).…”

“…In the present study, the cognitive approach was used in the three types of presentations as participants used their listening and reading comprehension skills to understand language presented through the teacher's narration and the written text. Dubois and Vial (2000) investigated the interaction between verbal and visual modes of presenting the foreign language for Russian vocabulary learning by French speaking learners. They predicted that their students would show better recall when textual information was presented with visual and auditory information as with semantic and phonetic links between the elements.…”

Vocabulary Instruction Through Books Read in American Sign Language for English-Language Learners With Hearing Loss

“…One population that is at risk and comprises ELLs is the population of children who are Hispanic and deaf or hard of hearing. Incorporating multimedia tools is effective in promoting vocabulary instruction of foreign languages if the presentation mode is matched to the task (Blom-Hoffman et al, 2006;Dubois & Vial, 2000). In two studies, computer-animated tutors increased vocabulary acquisition and fostered independence in students (Chambers, Abrami, McWhaw, & Therrien, 2001;Massaro & Light, 2004).…”

The Effects of Different Presentation Modalities on a Person Identification Task: A Study of Wireless Multimedia PDA Use in a Security Context

“…They suggest that information should be organized according to its perceived importance to the user, who should also have flexible information access at different levels of abstraction. Research on interaction between sound, written words, and the image of objects shows that when different sources of information are integrated, a learner's cognitive overload remains light and does not limit learning (Dubois & Vial, 2000). Andres and Petersen (2002) suggest that multimedia presentations invoke the use of both the verbal and visual working memory channels, resulting in a reduction of the cognitive load imposed by increased information complexity.…”