Coh-Metrix Model-Based Automatic Assessment of Interpreting Quality

Ouyang, Lingwei; Lv, Qianxi; Liang, Junying

doi:10.1007/978-981-15-8554-8_9

Cited by 9 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[17]. The documented versions of Coh-Metrix exist for Spanish [24], Portuguese [25], and Chinese [22] (however, they are developed independently and not supported by the initial authors).…”

Section: Related Studiesmentioning

confidence: 99%

The Grammar and Syntax Based Corpus Analysis Tool For The Ukrainian Language

Stetsenko,

Okulska

2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

This paper provides an overview of a corpus analysis tool -the StyloMetrix for the Ukrainian language. The StyloMetrix incorporates 104 metrics that cover grammatical, stylistic, and syntactic patterns.The idea of constructing the statistical evaluation of syntactic and grammar features is straightforward and familiar for the languages like English, Spanish, German, and others; it is yet to be developed for low-resource languages like Ukrainian. We describe the StyloMetrix pipeline and provide some experiments with this tool for the text classification task. We also describe our package's main limitations and the metrics' evaluation procedure.

show abstract

“…[17]. The documented versions of Coh-Metrix exist for Spanish [24], Portuguese [25], and Chinese [22] (however, they are developed independently and not supported by the initial authors).…”

Section: Related Studiesmentioning

confidence: 99%

The Grammar and Syntax Based Corpus Analysis Tool For The Ukrainian Language

Stetsenko,

Okulska

2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

show abstract

“…Specifically, it investigates "co-referential cohesion, causal cohesion, and density of connectives, latent semantic analysis metrics, and syntactic complexity" (Graesser & McNamara, 2011) by integrating "lexicons, pattern classifiers, part-of-speech taggers, syntactic parsers, shallow semantic interpreters, and other components that have been developed in the field of computational linguistics (Jurafsky & Martin, 2002)." This tool has been tested in second language proficiency and interpreting studies, which have generated fairly reliable results (Azadnia et al, 2019;Ouyang et al, 2021).…”

Section: Register Analysis Model Of the Source English Speechesmentioning

confidence: 99%

Multimodal register equivalence in student interpreters’ consecutive interpreting assessment

Xie

2022

ijtis

View full text Add to dashboard Cite

The present study proposes a source speech register analysis model that incorporates textual and situational components and parameters of register equivalence in the target language to identify the relations between the source language register and student interpreter’s performance, textual register mismatch between the source and target languages, and the relevance between register equivalence parameters in the target language and interpreting quality. The findings are as follows: first, source language register does have influence on interpreter’s performance, and relations exist between the two. In general, higher source language register brings better interpreting quality. Three assessment criteria and the overall quality differ across different levels of registers in the source language. Second, register mismatch between the source and target languages happens since interpreters are inclined to lower the register of interpreting output from the high-register source speech and increase the register level in the target language in the case of the low-register speech. Third, intonation and voice have the highest correlations with the interpreters’ performance. Fluency ranks the second, and lexical-semantic choices the third.

show abstract

“…In terms of automatic assessment based on linguistic/surface features, we highlight two recent explorations (Liu, 2021;Ouyang et al, 2021) which conducted corpus-based computational linguistic analysis to characterise transcribed interpreting texts, and to correlate indices of linguistic features with human scoring. 4 In Liu's ( 2021) study, 64 English-to-Chinese consecutive interpreting samples were selected from the Parallel Corpus of Chinese EFL Learners (PACCEL), a corpus consisting of more than 2 million words for Chinese-English translation and interpreting data.…”

Section: Automatic Assessment Of Interpreting: Recent Research Developmentsmentioning

confidence: 99%

Interpreting quality assessment re-imagined: The synergy between human and machine scoring

Han

2021

Interpreting and Society

View full text Add to dashboard Cite

Assessment of interpreting quality is a ubiquitous social practice in the interpreting industry and academia. In this article, we focus on both psychometric and social dimensions of assessment practice, and analyse two major assessment paradigms, namely, human rater scoring and automatic machine scoring. Regarding human scoring, we describe five specific methods, including atomistic scoring, questionnaire-based scoring, multi-methods scoring, rubric scoring, and ranking, and critically analyse their respective strengths and weaknesses. In terms of automatic scoring, we highlight four assessment approaches that have been researched and operationalised in cognate disciplines and interpreting studies, including automatic assessment based on temporal variables, linguistic/surface features, machine translation metrics, and quality estimation methodology. Finally, we problematise the socio-technological tension between these two paradigms and envisage human–machine collaboration to produce psychometrically sound and socially responsible assessment. We hope that this article sparks more scholarly discussion of rater-mediated and automatic assessment of interpreting quality from a psychometric-social perspective.

show abstract

Coh-Metrix Model-Based Automatic Assessment of Interpreting Quality

Cited by 9 publications

References 38 publications

The Grammar and Syntax Based Corpus Analysis Tool For The Ukrainian Language

The Grammar and Syntax Based Corpus Analysis Tool For The Ukrainian Language

Multimodal register equivalence in student interpreters’ consecutive interpreting assessment

Interpreting quality assessment re-imagined: The synergy between human and machine scoring

Contact Info

Product

Resources

About