Syntactic complexity measures as linguistic correlates of proficiency level in learner Russian

Kisselev, Olesya; Aleksandr, Klimov; Копотев, Михаил

doi:10.1075/scl.104.03kis

Cited by 3 publications

(2 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the structure of the Russian language is well described in theoretical literature (Bailyn, 2012; see also http://rusgram.ru/), scholarship on register‐ and genre‐based characteristics of the Russian language is very limited (Kisselev et al., 2022). Researchers of Russian academic genre writing (which may stand as a proxy for advanced language proficiency) highlight the preponderance of complex (mostly subordinated) sentences (Barykina et al., 1978; Khimik, 2003; Lobanova & Slesareva, 1980), as well as frequent use of participles and gerunds (Khimik, 2003).…”

Section: Linguistic Complexitymentioning

confidence: 99%

“…In a follow‐up study that used the same corpus data while applying a new set of computational analyses, Kisselev et al. (2022) found clear trends in the complexification of syntax between the intermediate and advanced levels. Specifically, there was a statistically significant increase in the mean length of a sentence for both the L2 and the heritage groups; interestingly, the study found an effect of language background on the mean length of a sentence.…”

Section: Linguistic Complexitymentioning

confidence: 99%

See 1 more Smart Citation

Measuring Writing Development and Proficiency Gains Using Indices of Lexical and Syntactic Complexity: Evidence From Longitudinal Russian Learner Corpus Data

Kisselev

SOYAN

PASTUSHENKOV

et al. 2022

The Modern Language Journal

View full text Add to dashboard Cite

Linguistic complexity serves as an important measure of second language (L2) writing development. Complexity indices, however, rarely feature in the studies of learner languages other than English. Additionally, L2 writing studies have been criticized for the lack of consistency in defining proficiency. The current study addresses these gaps by exploring writing development in a less commonly studied language (i.e., Russian) while paying attention to the operationalization of proficiency. We analyzed lexical and syntactic complexity indices in a corpus of essays written by learners at different levels at the beginning and the end of an instructional program. Our analyses demonstrate that 9 complexity indices (i.e., mean word length, type–token ratio, percentage of high‐frequency words, mean sentence length, clauses per sentence, syntactic depth, proportion of subordinate clauses, and proportion of relative clauses) changed significantly over the course of the program; the same 9 indices showed significant correlation with the initial curricular placement, and all—with the exception of clauses per sentence—modestly or highly correlated with the final rating on the proficiency test. The findings confirm the usefulness of complexity measures in tracking writing development in L2 Russian.

show abstract

Section: Linguistic Complexitymentioning

confidence: 99%

Section: Linguistic Complexitymentioning

confidence: 99%

Measuring Writing Development and Proficiency Gains Using Indices of Lexical and Syntactic Complexity: Evidence From Longitudinal Russian Learner Corpus Data

Kisselev

SOYAN

PASTUSHENKOV

et al. 2022

The Modern Language Journal

View full text Add to dashboard Cite

show abstract

Korean Syntactic Complexity Analyzer (KOSCA): An NLP application for the analysis of syntactic complexity in second language production

Hwang,

Kim

2024

Language Testing

View full text Add to dashboard Cite

Given the lack of computational tools available for assessing second language (L2) production in Korean, this study introduces a novel automated tool called the Korean Syntactic Complexity Analyzer (KOSCA) for measuring syntactic complexity in L2 Korean production. As an open-source graphic user interface (GUI) developed in Python, KOSCA provides seven indices of syntactic complexity, including traditional and Korean-specific ones. Its validity was tested by investigating whether the syntactic complexity indices measured by it in L2 written and spoken production could explain the variability of L2 Korean learners’ proficiency. The results of mixed-effects regression analyses showed that all seven indices significantly accounted for learner proficiency in Korean. Subsequent stepwise multiple regression analyses revealed that the syntactic complexity indices explained 56.0% of the total variance in proficiency for the written data and 54.4% for the spoken data. These findings underscore the validity of the syntactic complexity indices measured by KOSCA as reliable indicators of L2 Korean proficiency, which can serve as a valuable resource for researchers and educators in the field of L2 Korean learning and assessment.

show abstract

Exploring collocational complexity in L2 Russian: A corpus-driven contrastive analysis

Kopotev,

Klimov,

Kisselev

2023

International Journal of Bilingualism

View full text Add to dashboard Cite

Objective: The objective of this article is to discuss the pedagogical and practical need for automated assessment tools that enable teachers, researchers, and other language practitioners to relatively quickly and automatically assess the general proficiency of second language (L2) speakers according to a number of different linguistic parameters, specifically the use of collocations. Introduction: The Introduction discusses existing approaches to the concept of collocation and its role in L2 corpus studies. Methods: In the main part, Toward the automatic detection of collocational strength, we describe an approach to assessing collocations in Russian L2 written production, which is based on the corpus-driven contrastive analysis. In brief, this approach proposes ranking L2 collocations in accordance with the “collocational scores” of bigrams extracted from a reference (L1) corpus, with the assumption that the greater number of native-like bigrams in an L2 text and the higher collocational scores associated with them, the more proficient is the learner who created the text. To test this hypothesis, we conducted a series of experiments based on an L1 reference corpus and split the L2 datasets into four proficiency groups (Beginner, Intermediate, Advanced, and Superior). Results and evaluation: The findings, presented in the section Results and evaluation of the methods, established an empirical minimum and maximum of mean reciprocal rank (MRR) values, and the variation of the MRR across the four proficiency levels. In the last sections, we discuss shortcomings of the methods and propose further steps for overcoming them. We believe that the proposed method for assessing the collocational skill of Russian L2 writers can be successfully integrated into a bigger system capable of multifaceted automatic assessment of Russian learner texts.

show abstract

Syntactic complexity measures as linguistic correlates of proficiency level in learner Russian

Cited by 3 publications

References 52 publications

Measuring Writing Development and Proficiency Gains Using Indices of Lexical and Syntactic Complexity: Evidence From Longitudinal Russian Learner Corpus Data

Measuring Writing Development and Proficiency Gains Using Indices of Lexical and Syntactic Complexity: Evidence From Longitudinal Russian Learner Corpus Data

Korean Syntactic Complexity Analyzer (KOSCA): An NLP application for the analysis of syntactic complexity in second language production

Exploring collocational complexity in L2 Russian: A corpus-driven contrastive analysis

Contact Info

Product

Resources

About