The Dual-Coding and Multimedia Learning Theories: Film Subtitles as a Vocabulary Teaching Tool

Abstract

The use of multimedia has often been suggested as a teaching tool in foreign language teaching and learning. In foreign language education, exciting new multimedia applications have appeared over the last years, especially for young learners, but many of these do not seem to produce the desired effect in language development. This article looks into the theories of dual-coding (DCT) and multimedia learning (CTML) as the theoretical basis for the development of more effective digital tools with the use of films and subtitling. Bilingual dual-coding is also presented as a means of indirect access from one language to another and the different types of subtitling are explored regarding their effectiveness, especially in the field of short-term and long-term vocabulary recall and development. Finally, the article looks into some new alternative audiovisual tools that actively engage learners with films and subtitling, tailored towards vocabulary learning.The dual-coding theory (DCT) is a general cognition theory that has been directly applied to literacy and language learning. The theory was proposed by Allan Paivio in 1971 and explains the powerful effects of mental imagery on the mind and memory. In his theory, Paivio originally accounted for verbal and nonverbal influences in memory, but researchers soon started applying it in other areas of cognition [8][9][10]. According to this theory, a person can learn new materials using verbal associations or visual imagery but the combination of both is more successful in learning [11]. The dual-coding theory states that the brain uses both visual and verbal information to represent information [12], but this information is processed differently along two distinct channels in the human mind, creating different representations for information that each channel processes. The existing two coding systems are the verbal system and the nonverbal/visual system. These two coding systems interact, and this interaction results in better recall [10,13]. The verbal system stores linguistic information/units (such as text, sound, or even motor experience such as sign language) in sequential units called "logogens." The non-verbal/visual system processes visual information/units (such as symbols, pictures, or videos) and keeps them in units called "imagens." The terms "logogen" and "imagen" refer respectively to representational units of verbal and nonverbal information that produce already existing mental words and images and can function unconsciously to improve cognitive performance [14]. According to Paivio [9] and Clark and Paivio [15], there are three different processing levels that take place within or between verbal and nonverbal/visual systems: representational, referential, and associative processing. The two systems are linked together through referential connections (Figure 1).

Video Cartoons and Task-Induced Involvement: Effects to Pupils’ L2 Incidental Literacy Acquisition

“…The findings on the high frequency words with more children acquiring them across the three skills corroborates the Dual-Coding Theory by Paivio (2006) which state that these units presented as verbal and non-verbal cues in audio-visual input can function unconsciously to improve cognitive performance (Paivio, 2014 as cited in Kanellopoulou, Kermanidis and Giannakoulopoulos, 2019). The non-verbal and verbal codes occurring frequently may have positive impacts on recall (Kanellopoulou, Kermanidis and Giannakoulopoulos, 2019). This supports Ambridge, Kidd, Rowland & Theakston (2015) claim that "All other things being equal, frequent forms will be acquired before less-frequent forms" (p.241).…”

“…In other words, correlations and interconnectedness exist within test scores of one literacy skill and across test scores of other literacy skills. The findings on the high frequency words with more children acquiring them across the three skills corroborates the Dual-Coding Theory by Paivio (2006) which state that these units presented as verbal and non-verbal cues in audio-visual input can function unconsciously to improve cognitive performance (Paivio, 2014 as cited in Kanellopoulou, Kermanidis and Giannakoulopoulos, 2019). The non-verbal and verbal codes occurring frequently may have positive impacts on recall (Kanellopoulou, Kermanidis and Giannakoulopoulos, 2019).…”

“…Anchored on the Dual Coding Theory (DCT) by Paivio (2006), this study examines how the verbal, nonverbal and textual cues affect the learners' SLA in terms of comprehension, vocabulary, orthography, and pronunciation. The DCT has been directly used in literacy and language learning as a general cognition theory (Kanellopoulou, Kermanidis and Kanellopoulou, 2019). According to Paivio (2006), the Dual Coding Theory involves the two distinct subsystems activity-the verbal and the nonverbal system.…”

“…Paivio (2006) further adds that these two systems are composed of internal representational units, which he calls logogens and imagens, which are both activated when one recognizes, manipulates, or thinks of words and things. Hence, these units can function unconsciously to improve cognitive performance (Paivio, 2014 as cited in Kanellopoulou, Kermanidis and Giannakoulopoulos, 2019). The representations of these units are modality-specific (Paivio, 2006) so every visual, auditory, haptic or feel and motor properties of language that one encounters correspond to a different logogen and imagen.…”

Educación y medios audiovisuales: una reflexión sistémica para su implementación, fortalecimiento y sostenibilidad

“…Un aspecto relacionado con la idea de calidad es la idea de que el producto que se ofrece a los estudiantes resulta más efectivo cuando se puede visionar y oír íntegramente. Si bien resulta pertinente el uso de la subtitulación para la enseñanza de lenguas, y como soporte para estudiantes con problemas auditivos (Kanellopoulou, Kermanidis & Giannakoulopoulos, 2019), el grado de atención que provoca un MA es, en general, mucho mayor cuando se combinan ambos elementos sensoriales. Tal es el caso de la propuesta de Xie, Mayer, Wang y Zhou (2019), en un experimento que evaluó los efectos conseguidos en una versión subtitulada y una versión con audio.…”