The impact of keyword and full video captioning on listening comprehension

Abstract

This study investigates the effect of two types of captioning (full and keyword captioning) on listening comprehension. Thirty-six university-level EFL students participated in the study. They were randomly assigned to watch three video clips under three conditions. The first group watched the video clips with full captions. The second group watched the same video clips with keyword captions. The control group watched the video clips without captions. After watching each clip, participants took a listening comprehension test. At the end of the experiment, participants completed a questionnaire to measure their perceptions about the use of captions and the video clips they watched. Results indicated that the full captioning group significantly outperformed both the keyword captioning and the no captioning group on the listening comprehension tests. However, this study did not find any significant difference between the keyword captioning group and the no captioning group. Results of the survey suggest that keyword captioning were a source of distraction for participants.

Automatic Compression of Subtitles with Neural Networks and its Effect on User Experience

“…However, there is no significant difference between human compressed and system compressed subtitles (p corrected > 0.99). These higher cognitive load scores for partial subtitles are in line with the findings of Montero Perez et al [23], Behroozizad and Majidi [24] and Bensalem [25]. This could be explained by a lack of "belongingness" (c.f.…”

“…They argue based on the dual-coding theory that as they lower the input on the visual channel, the cognitive load has to be smaller. Others, however, report worse results with keyword captions compared to full captions or no captions due to confusion and distraction [23,24,25]. These results stress the need of additional user studies, as mere technical evaluations cannot provide enough insights on how the chosen technological methods affect the user.…”

The impact of subtitles and transcripts in disciplinary videos for EAL learners

“…Guided by the cognitive theory of multimedia learning (CTML) model (Mayer, 2014), previous design-based research has invested a significant amount of effort in examining the effectiveness of language supporting tools in instructional videos, such as subtitles, captions, narrations, and transcripts to support EAL learners (Bensalem, 2017;Liao et al, 2020;Teng, 2020). However, due to the existence of various boundary conditions found in general research on CTML principles (Fyfield et al, 2022), and the lack of learner-centered studies around non-English speaking students, there remains a compelling need for further research to capture the nuanced interactions between the language factors, cultural backgrounds, and instructional designs of videos.…”