Lexical Features of Text Complexity: the case of Russian academic texts

Churunina, Anna A.; Solnyshkina, Marina; Gafiyatova, Elzara V.; Заикин, А. А.

doi:10.1051/shsconf/20208801009

Cited by 2 publications

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Упоминаемые в различных работах признаки чрезвычайно разнообразны. В исследованиях, посвящённых оценке сложности русскоязычных текстов, помимо традиционно используемых признаков, таких как средние длины слов и предложений, встречается использование оценок лексического разнообразия [4][5][6], морфологических [5][6][7][8][9], синтаксических [6,[8][9][10], лексических [6-9, 11, 12], семантических [9], тематических [6] признаков.…”

Section: Introductionunclassified

Keywords, morpheme parsing and syntactic trees: features for text complexity assessment

Morozov,

Smal,

Garipov

et al. 2024

Model. anal. inf. sist.

View full text Add to dashboard Cite

The text complexity assessment is an applied problem of current interest with potential application in the drafting of legal documents, editing textbooks, and selecting books for extracurricular reading. The methods for generating a feature vector when automatically assessing the text complexity are quite diverse. Early approaches relied on easily calculable quantities, such as the average length of a sentence or the average number of syllables per word. With the development of natural language processing algorithms, the space of used features is expanding. In this work, we examined three groups of features: 1) automatically generated keywords, 2) information about the features of morphemic word parsing, and 3) information about the diversity, branching, and depth of syntactic trees. The RuTermExtract algorithm was utilized to generate keywords, a convolutional neural network model was used to generate morphemic parses, and the Stanza model, trained on the SynTagRus corpus, was used to generate syntax trees. We conducted a comparison using four different machine learning algorithms and four annotated Russian-language text corpora. The corpora used differ both in the domain and markup paradigm, due to which the results obtained more objectively reflect the real relationship between the characteristics and the text complexity. The use of keywords performed worse on average than the use of topic markers obtained using latent Dirichlet allocation. In most situations, morphemic characteristics turned out to be more effective than previously described methods for assessing the lexical complexity of a text: the frequency of words and the occurrence of word-formation patterns. The use of an extensive set of syntactic features allowed, in most cases, to improve the quality of work of neural network models in comparison with the previously described set.

show abstract

Section: Introductionunclassified

Keywords, morpheme parsing and syntactic trees: features for text complexity assessment

Morozov,

Smal,

Garipov

et al. 2024

Model. anal. inf. sist.

View full text Add to dashboard Cite

show abstract

ReaderBench: Multilevel analysis of Russian text characteristics

Corlatescu

Ruşeţi²,

Dascălu³

2022

Russian Journal of Linguistics

View full text Add to dashboard Cite

This paper introduces an adaptation of the open source ReaderBench framework that now supports Russian multilevel analyses of text characteristics, while integrating both textual complexity indices and state-of-the-art language models, namely Bidirectional Encoder Representations from Transformers (BERT). The evaluation of the proposed processing pipeline was conducted on a dataset containing Russian texts from two language levels for foreign learners (A - Basic user and B - Independent user). Our experiments showed that the ReaderBench complexity indices are statistically significant in differentiating between the two classes of language level, both from: a) a statistical perspective, where a Kruskal-Wallis analysis was performed and features such as the “nmod” dependency tag or the number of nouns at the sentence level proved the be the most predictive; and b) a neural network perspective, where our model combining textual complexity indices and contextualized embeddings obtained an accuracy of 92.36% in a leave one text out cross-validation, outperforming the BERT baseline. ReaderBench can be employed by designers and developers of educational materials to evaluate and rank materials based on their difficulty, as well as by a larger audience for assessing text complexity in different domains, including law, science, or politics.

show abstract

Lexical Features of Text Complexity: the case of Russian academic texts

Cited by 2 publications

References 10 publications

Keywords, morpheme parsing and syntactic trees: features for text complexity assessment

Keywords, morpheme parsing and syntactic trees: features for text complexity assessment

ReaderBench: Multilevel analysis of Russian text characteristics

Contact Info

Product

Resources

About