Effects of modality preference and working memory capacity on captioned videos in enhancing L2 listening outcomes

Abstract

Captioned video is widely used to enhance second language (L2) learners’ exposure to oral input beyond the classroom setting, and captioning has been found to provide an instantaneous, useful visual aid for parsing and understanding L2 oral discourse. Nevertheless, a meta-analysis has shown that captioning exerts a selective effect on L2 learners with different profiles. This study investigated whether L2 learners’ modality preferences (visual vs. auditory) and working memory capacity (high vs. low) would modulate the effect of full captions on L2 listening outcome. Results from 60 participants revealed that both cognitive variables affected their L2 listening to different extents. Notably, working memory capacity modulates the impact of L2 learners’ preferred modality on their listening outcome. Modality preference did not exert any significant impact on the listening outcome of L2 learners with lower working memory capacity. For L2 learners with high working memory capacity, their modality preference played a pivotal role in modulating their listening outcome; in this case, auditory learners had the best listening performance viewing the video without captions, whereas visual learners did best when watching the captioned video. These findings speak to the need for taking individual differences into consideration when employing captioned videos.

Dual Coding or Cognitive Load? Exploring the Effect of Multimodal Input on English as a Foreign Language Learners’ Vocabulary Learning

“…Although some studies (e.g., Taylor, 2005;Kam et al, 2020) have shown that learners do not necessarily have preferences for multimodal input, whose additional information may devolve as attention-getter and, thus, detrimental to vocabulary learning, positive perceptions of the learners benefiting from the advantages of multimodal input in this study have been previously identified by some scholars. Shadiev et al (2020), for instance, developed a vocabulary learning system featuring image-to-text recognition (ITR) technology.…”

The why-what-when-who-how of using captioned videos as an instructional aid in EAL classrooms: Theoretical perspectives and classroom implications

“…Similarly, Vanderplank (2016) noted that captions help viewers to improve segmentation speech processing, that is, the ability to identify word boundaries and language chunks. Additionally, Kam, Liu, and Tseng (2020) studied the influences of working memory capacity and modality preferences on captioned viewings to enhance L2 listening performances. The findings first showed that captions were beneficial for processing L2 listening comprehension.…”

The mediating role of vocabulary size, working memory, attention and inhibition in early vocabulary learning under different TV genres: An exploratory study

“…Evidence from empirical studies measuring both executive control and storage have been inconclusive, with studies showing a large role of WM (e.g., Kormos & Sáfár, 2008) and others showing no role of WM (e.g., Juffs, 2005) in L2 performance. More recently, it has been found that high working memory learners benefit from the modality of their preference in listening outcomes when exposed to captioned and uncaptioned video materials (Kam et al., 2020).…”

“…Even fewer have looked at how learners' viewing experience is affected by individual differences (IDs) in vocabulary size, attention, inhibition, or working memory. Such IDs have been shown to mediate early and long-term vocabulary learning (Gass, Winke, Isbell, & Ahn, 2019;Kam, Liu, & Tseng, 2020;Peters & Webb, 2018). This exploratory study seeks to analyze how vocabulary is acquired under different genres and how this process may be influenced by the IDs mentioned above.…”