Processing of native and foreign language subtitles in films: An eye tracking study

Bisson, Marie Josée; Heuven, Walter J. B. van; Conklin, Kathy; Tunney, Richard J.

doi:10.1017/s0142716412000434

Cited by 128 publications

(112 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hinkin et al 2014). It has also been found that regardless of their type -be they standard interlingual (foreign language audio with native language subtitles), intralingual (audio and subtitles in the same language, foreign or native), or reversed (native language audio with 6 foreign language subtitles) -subtitles constitute a major gaze attractor (Bisson, Van Heuven, Conklin and Tunney 2014;Kruger, Szarkowska and Krejtz 2015). Although people follow subtitles for a significant share of their presentation time (d 'Ydewalle, Muylle and van Rensbergen 1985;Jensema, Danturthi and Burch 2000), the influence of the language of the soundtrack on subtitle processing has so far yielded contradictory results.…”

Section: Introductionmentioning

confidence: 99%

Is subtitling equally effective everywhere? A first cross-national study on the reception of interlingually subtitled messages

Perego¹,

Laskowska²,

Matamala³

et al. 2016

Across Languages and Cultures

View full text Add to dashboard Cite

Recent research on the reception of interlingual subtitling revealed that it is cognitively effective: watching a subtitled film results in a good understanding of the film content, it does not require a significant tradeoff between image processing and text processing, and it leads to a good performance in the recognition of the words and expressions 4 contained in the subtitles. To date, the studies that revealed the effectiveness of subtitle processing have been conducted mono-nationally -e.g. d 'Ydewalle and De Bruycker (2007) in Belgium; Wissmath et al. (2009) in Switzerland;Perego et al. (2010Hinkin et al. (2014) in the US. However, it has not yet been demonstrated empirically whether subtitle effectiveness varies depending on the familiarity of viewers with subtitles. The crossnational study described in this paper aims to fill this gap and appraise the cognitive performance and overall appreciation of a moderately complex subtitled film by viewers with different degrees of familiarity with subtitles, i.e., viewers living in countries (Italy, Spain, Poland and Dutch-speaking Belgium) with different audiovisual translation traditions. The main findings reveal that subtitling is effective irrespective of users' familiarity with it, although it is not enjoyed equally among the tested populations.

show abstract

Section: Introductionmentioning

confidence: 99%

Is subtitling equally effective everywhere? A first cross-national study on the reception of interlingually subtitled messages

Perego¹,

Laskowska²,

Matamala³

et al. 2016

Across Languages and Cultures

View full text Add to dashboard Cite

show abstract

“…In a follow-up study, Bisson, van Heuven, Conklin, and Tunney (2014b) varied the number of exposures (2, 4, 6, and 8) to the FL words during the letter-search task. The results showed that as little as two exposures to words in the incidental learning task was sufficient for incidental vocabulary acquisition to occur.…”

mentioning

confidence: 99%

“…In the two studies from Bisson et al (2013, 2014b). FL acquisition occurred when both the auditory and written word forms presented during the incidental learning phase were in the FL.…”

mentioning

confidence: 99%

“…With standard subtitles, one can enjoy the visual aspects of the film and the original soundtrack and understand the story through the NL subtitles. Although it is clear that people read NL subtitles (Bisson, van Heuven, Conklin, & Tunney, 2014a; d'Ydewalle & de Bruycker, 2007), it is less certain whether the FL words in the soundtrack are also processed. Since the subtitles provide a NL translation of the dialogue, there is no need to attend to the FL words.…”

mentioning

confidence: 99%

“…In the multimodal situation used in Bisson et al (2013, 2014b), words were repeated from 2 to 8 times only, and this was sufficient for FL vocabulary acquisition to occur. Participants were able to recognize the correct translation equivalent of the FL auditory words in a test phase, suggesting that even irrelevant auditory information was processed during an incidental learning phase.…”

mentioning

confidence: 99%

See 2 more Smart Citations

The role of verbal and pictorial information in multimodal incidental acquisition of foreign language vocabulary

Bisson

Heuven

Conklin

et al. 2015

Quarterly Journal of Experimental Psychology

Self Cite

View full text Add to dashboard Cite

This study used eye tracking to investigate the allocation of attention to multimodal stimuli during an incidental learning situation, as well as its impact on subsequent explicit learning. Participants were exposed to foreign language (FL) auditory words on their own, in conjunction with written native language (NL) translations, or with both written NL translations and pictures. Incidental acquisition of FL words was assessed the following day through an explicit learning task where participants learned to recognize translation equivalents, as well as one week later through recall and translation recognition tests. Results showed higher accuracy scores in the explicit learning task for FL words presented with meaning during incidental learning, whether written meaning or both written meaning and picture, than for FL words presented auditorily only. However, participants recalled significantly more FL words after a week delay if they had been presented with a picture during incidental learning. In addition, the time spent looking at the pictures during incidental learning significantly predicted recognition and recall scores one week later. Overall, results demonstrated the impact of exposure to multimodal stimuli on subsequent explicit learning, as well as the important role that pictorial information can play in incidental vocabulary acquisition.

show abstract

Subtitles and Eye Tracking: Reading and Performance

Kruger

Steyn

2013

Read Res Q

View full text Add to dashboard Cite

A B S T R A C TThis article presents an experimental study to investigate whether subtitle reading has a positive impact on academic performance. In the absence of reliable indexes of reading behavior in dynamic texts, the article first formulates and validates an index to measure the reading of text, such as subtitles on film. Eye-tracking measures (fixations and saccades) are expressed as functions of the number of standard words and word length and provide a reliable index of reading behavior of subtitles over extended audiovisual texts. By providing a robust index of reading over dynamic texts, this article lays the foundation for future studies combining behavioral measures and performance measures in fields such as media psychology, educational psychology, multimedia design, and audiovisual translation. The article then utilizes this index to correlate the degree to which subtitles are read and the performance of students who were exposed to the subtitles in a comprehension test. It is found that a significant positive correlation is obtained between comprehension and subtitle reading for the sample, providing some evidence in favor of using subtitles in reading instruction and language learning. The study, which was conducted in the context of English subtitles on academic lectures delivered in English, further seems to indicate that the number of words and the number of lines do not play as big a role in the processing of subtitles as previously thought but that attention distribution across different redundant sources of information results in the partial processing of subtitles.A s digital contexts proliferate, the types of texts that children and adults encounter keep changing. Moreover, as the contexts in which reading occurs change, the reader has to adopt a variety of reading styles. These reading styles have to adapt to static text on a stable background on the one end of the spectrum (where the pace of reading is determined by the reader) and fleeting text on a dynamic background, as in subtitled or captioned audiovisual texts, on the other end of the spectrum (where the reader has to adjust the pace of reading to the pace of presentation). Furthermore, new media such as smartphones and tablets present information in increasingly hands-free modes that anticipate reading behavior by means of algorithms and scroll text (semi)automatically, presenting readers with a combination of dynamic and static texts.It can be assumed that the exposure to and reliance on dynamic texts, such as video and interactive multimedia, in both formal and informal education will continue to increase. This could have farreaching effects for reading instruction if subtitles and other text were to be introduced in a more conscious manner. However, before teachers and reading instructors can employ modes such as subtitles optimally, we require more insight into the way in which children and

show abstract

Processing of native and foreign language subtitles in films: An eye tracking study

Cited by 128 publications

References 36 publications

Is subtitling equally effective everywhere? A first cross-national study on the reception of interlingually subtitled messages

Is subtitling equally effective everywhere? A first cross-national study on the reception of interlingually subtitled messages

The role of verbal and pictorial information in multimodal incidental acquisition of foreign language vocabulary

Subtitles and Eye Tracking: Reading and Performance

Contact Info

Product

Resources

About