Viewers can keep up with fast subtitles: Evidence from eye movements

Abstract

People watch subtitled audiovisual materials more than ever before. With the proliferation of subtitled content, we are also witnessing an increase in subtitle speeds. However, there is an ongoing controversy about what optimum subtitle speeds should be. This study looks into whether viewers can keep up with increasingly fast subtitles and whether the way people cope with subtitled content depends on their familiarity with subtitling and on their knowledge of the language of the film soundtrack. We tested 74 English, Polish and Spanish viewers watching films subtitled at different speeds (12, 16 and 20 characters per second). The films were either in Hungarian, a language unknown to the participants (Experiment 1), or in English (Experiment 2). We measured viewers’ comprehension, self-reported cognitive load, scene and subtitle recognition, preferences and enjoyment. By analyzing people’s eye gaze, we were able to discover that most viewers could read the subtitles as well as follow the images, coping well even with fast subtitle speeds. Slow subtitles triggered more re-reading, particularly in English clips, causing more frustration and less enjoyment. Faster subtitles with unreduced text were preferred in the case of English videos, and slower subtitles with text edited down in Hungarian videos. The results provide empirical grounds for revisiting current subtitling practices to enable more efficient processing of subtitled videos for viewers.

Two or three lines: a mixed-methods study on subtitle processing and preferences

“…Despite the differences in the self-reported cognitive load, enjoyment and preferences, comprehension was not found to be affected by the number of lines. This is in line with some previous studies, which also found that the manipulation of various subtitle parameters did not impact on comprehension, as was the case in the study by Perego et al (2010) on subtitle segmentation, and two studies on subtitle speeds by Szarkowska et al (2011) and Szarkowska and Gerber-Morón (2018b). The lack of differences in comprehension scores between the conditions may be attributed to viewers' previous exposure to different types of subtitlingboth high-quality subtitles fully conforming to the standards and poor-quality subtitles not necessarily compliant with all subtitling rules.…”

“…Interestingly, the values reported in these studies are substantially lower compared to our study, in which participants looked at the subtitles for about half of their display time. These differences may be attributed to a fast pace of contemporary productions and to the speed of subtitles (for more on subtitle speed see Romero-Fresco (2015) and Szarkowska and Gerber-Morón (2018b). In our study, participants spent slightly more time in the three-line compared to the two-line condition, which means that they had relatively less time to follow the on-screen action.…”

“…Drawing largely on the eye-mind hypothesis, according to which "there is no appreciable lag between what is fixated and what is processed" (Just & Carpenter, 1980, p. 331), eye tracking has been used as an online measure of visual attention allocation in subtitle processing, allowing researchers "unprecedented access into the eyes, and arguably the minds, of the viewers" (Doherty & Kruger, 2018). In recent years, eye tracking has been used to study different aspects of subtitling: line breaks and segmentation Gerber-Morón, Szarkowska, & Woll, 2018;Perego, Del Missier, Porta, & Mosconi, 2010;Rajendran, Duchowski, Orero, Martínez, & Romero-Fresco, 2013), text editing and reading speed (d 'Ydewalle, Rensbergen, & Pollet, 1987;Jensema, Danturthi, & Burch, 2000;Koolstra, Van Der Voort, & d'Ydewalle, 1999;Szarkowska & Gerber-Morón, 2018b;Szarkowska, Krejtz, Pilipczuk, Dutka, & Kruger, 2016), shot changes (Krejtz, Szarkowska, & Krejtz, 2013), the impact of sound and type of subtitling on subtitle processing (Bisson, Van Heuven, Conklin, & Tunney, 2012), cognitive processing (Kruger & Doherty, 2016;Kruger, Hefer, & Matthew, 2014), and types of viewers (d' Ydewalle & De Bruycker, 2007;De Bruycker & d'Ydewalle, 2003;Matamala, Perego, & Bottiroli, 2017;Perego, Del Missier, & Stragà, 2018;Perego et al, 2016). The increasing body of research has shown that the processing of subtitles not only depends on the type of subtitles, but is also conditioned by the characteristics of the audience, including their linguistic abilities or hearing status (d' Ydewalle & De Bruycker, 2007;De Bruycker & d'Ydewalle, 2003;De Linde & Kay, 1999;Krejtz, Szarkowska, & Łogińska, 2016;Łuczak, 2017;Muñoz, 2017;Szarkowska et al, 2016;Winke, Gass, & Sy...…”

“…Yet, owing to the time limitations and the nature of the study, it was not possible for us to test participants' language proficiency directly. Participants were given a sheet (Szarkowska & Gerber-Morón, 2018b) describing the skills required at each proficiency level using the Common European Framework of Reference for Languages. They made their assessment based on the sheet as well as their own experience with the European system which they were familiar with from their education in schools and at university.…”

Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences

“…Recent research conducted with hearing viewers suggests that fast subtitle speeds, of 20 characters per second, are preferred to slower ones when the viewers are proficient in the language of the clip. If the language is unknown, slower speeds are preferred, though the researchers do not specify the value (Szarkowska and Gerber-Morón 2018).…”