Human-Informed Speakers and Interpreters Analysis in theWAWCorpus and an Automatic Method for Calculating Interpreters’ Décalage

Temnikova, Irina; Freelancer, Sofia; Abdelali, Ahmed; Djabri, Souhila; Hedaya, Samy

doi:10.26615/issn.2683-0078.2019_013

Cited by 3 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The breakthrough of NMT resulted in MT output that differed slightly from that of previous MT systems: it makes fewer errors than previous systems (Temnikova et al, 2019) and provides translations that are closer to being human-like in their fluency (Bhardwaj et al, 2020). Studies like the ones mentioned only measure the lack of fluency errors, and therefore, the human-like quality should be confirmed by exploring alternative quality evaluation methods.…”

Section: Related Workmentioning

confidence: 99%

Evaluating NMT using the non-inferiority principle

do Campo Bayón,

Sánchez-Gijón

2024

NLP

View full text Add to dashboard Cite

The aim of this article is to propose a new neural machine translation (NMT) evaluation method based on the non-inferiority principle. In order to do that, we evaluate raw machine translation (MT) in terms of naturalness, which for this research is defined as not just the lack of fluency errors but also meeting the linguistic expectations of Galician end users when reading original texts in Galician. Our main objective is, in the first place, to validate the new methodology presented in our previous study by evaluating an NMT engine from Spanish into Galician for the social media domain that was retrained with a new Twitter corpus. This new methodology and NMT engine were applied after analyzing the conclusions of a pilot survey conducted among Twitter users to evaluate their perception of tweets translated from Spanish into Galician with our NMT engine created with a corpus of tweets. As in our preliminary study, our aim is to propose a robust quality approximation method based on the reception parameters of end users’ perceptions. This new survey was conducted in December of 2022 with the participation of 228 Galician-speaking Twitter users. Among the main changes proposed are the inclusion of more information about the participant profile, so the non-inferiority principle can be also evaluated according to these parameters; the inclusion of a new typology of tweets, the threads; the provision of context by means of presenting the tweets in their original display as shown in the Twitter app; a change in the number of tweets evaluated and the number of different questionnaires; the change in the distribution of the questionnaires; and the inclusion of an error classification human evaluation conducted by professional linguists to correlate the findings. We will present the steps carried out following the conclusions of the pilot study, describe the new study’s design, analyze the new findings, and present the final conclusions regarding the engine and the evaluation method based on the non-inferiority principle. Finally, we will also provide some examples of the use of this new methodology in the translation industry.

show abstract

Section: Related Workmentioning

confidence: 99%

Evaluating NMT using the non-inferiority principle

do Campo Bayón,

Sánchez-Gijón

2024

NLP

View full text Add to dashboard Cite

show abstract

“…Examples range from international medical conferences open to doctors and patients as in the DIRSI Corpus (Bendazzoli 2012) to pre-and postfootball matches press conferences in the FOOTIE Corpus (Sandrelli 2012). Finally, there are conference interpreting corpora compiled by researchers who apparently acted as external observers or obtained recordings from the organizers, as is the case with the WAW corpus, which includes English-Arabic / Arabic-English simultaneous interpretations of conference presentations from three different events held in Qatar, for a total of 521 sessions (Abdelali et al 2018;Temnikova et al 2019).…”

Section: A Partial Overview Of Conference Interpreting Corporamentioning

confidence: 99%

Corpus studies in conference interpreting

Bendazzoli¹

2021

The Routledge Handbook of Conference Interpreting

View full text Add to dashboard Cite

Research into conference interpreting has taken advantage of corpus methods since the early developments of the corpus-based approach. Overviews of major projects in corpus-based interpreting studies clearly show that interpreting corpora have been increasing in number and size over the last three decades, covering a variety of language combinations along with multiple modes and settings. Overall, conference interpreting corpora range from small data sets used for qualitative "manual" analysis to larger, machine-readable corpora, some of which are also accessible online. The systematic observations afforded by corpus methods have allowed interpreting scholars to study different features of conference interpreting with respect to source speeches and other Translation modes, combining a variety of methods including statistical testing, comparable and parallel inquiries, and discourse analysis. Some pedagogical applications have also begun to find their way in conference interpreting education, while there is still very limited evidence of the role of corpora in conference interpreters' professional practice.

show abstract

From manual to machine

Guo,

Han

2024

INTP

View full text Add to dashboard Cite

This study introduces a groundbreaking automated methodology for measuring ear–voice span (EVS) in simultaneous interpreting (SI). Traditionally, assessing EVS – a critical temporal metric in SI – has been hampered by labour-intensive and time-consuming manual methods that are prone to inconsistency. To overcome these challenges, our research harnesses state-of-the-art natural language processing (NLP) technologies, including automatic speech recognition (ASR), sentence boundary detection (SBD) and cross-lingual alignment, to automate EVS measurement. We deployed a comprehensive array of NLP models and evaluated the automated pipelines on a 20-hour English-to-Portuguese SI corpus which featured 57 varied audio pairings. The findings are encouraging: the most effective model combination achieved a median EVS error of less than 0.1 seconds across the corpus. Moreover, the automated pipelines exhibited a high level of accuracy, strong correlation and substantial agreement with manual measurements when assessing median EVS for individual audio pairs. Despite these satisfactory results, certain challenges persist with some NLP models, indicating clear avenues for future research. This study not only introduces a groundbreaking approach to large-scale EVS measurement but also propels the automation of process analysis in Interpreting Studies.

show abstract

Human-Informed Speakers and Interpreters Analysis in theWAWCorpus and an Automatic Method for Calculating Interpreters’ Décalage

Cited by 3 publications

References 19 publications

Evaluating NMT using the non-inferiority principle

Evaluating NMT using the non-inferiority principle

Corpus studies in conference interpreting

From manual to machine

Contact Info

Product

Resources

About