Cognitive complexity measures for educational texts: Empirical validation of linguistic parameters

Abstract

The article presents a study conducted within the framework of discourse complexology - an integral scientific domain that has united linguists, cognitive scientists, psychologists and programmers dealing with the problems of discourse complexity. The issue of cognitive complexity of texts is one of the central issues in discourse complexology. The paper presents the results of the study aimed to identify and empirically validate a list of educational texts’ complexity predictors. The study aims to identify discriminant linguistic parameters sufficient to assess cognitive complexity of educational texts. We view text cognitive complexity as a construct, based on the amount of presented information and the success of reader-text interactions. The idea behind the research is that text cognitive complexity notably increases across middle and high schools. The research dataset comprises eight biology textbooks with the total size of 219,319 tokens. Metrics of text linguistic features were estimated with the help of automatic analyzer RuLingva (rulingva.kpfu.ru). Linguistic and statistical analysis confirmed the hypothesis that text syntactic and lexical parameters are discriminative enough to classify different levels of cognitive complexity of educational texts used in middle and high schools. Text parameters that manifest variance in cognitive complexity include lexical diversity (TTR); local argument overlap; abstractness index; number of polysyllabic words, Flesch-Kincaid Grade Level; number of nouns and number of adjectives per sentence. Empirical evidence indicates that the proposed approach outperforms existing methods of text complexity assessment. The research results can be implemented in the system of scientific and educational content expertise for Russian school textbooks. They can also be of some use in the development of educational resources and further research in the field of text complexity.

Variety and functional diversity of modern discourse in cognitive perspective

“…If the main achievement of text complexity studies in the XX th century was the idea that "different types of texts are complex in different ways" and "readability formulas are genre-dependent", modern discourse complexology addresses texts, cognitive aspects of reading and a level of challenge a text provides for readers. Based on a solid foundation of additive and non-additive types of complexity, researchers implement not only parametric methods measuring objective complexity of texts but criteria approach assessing relative complexity of texts for different categories of readers or individuals (Kupriyanov et al 2023.…”

“…Если главным достижением исследований в области сложности текста в XX в. была идея о том, что «разные типы текстов сложны по-разному» и «формулы читабельности зависят от жанра», то современная дискурсивная комплексология обращается не только к текстам, но когнитивным аспектам восприятия, обработки информации и понимания текста, а также к вопросам (не)готовности читателя к восприятию текста. Основываясь на прочном фундаменте аддитивных и неаддитивных типов сложности, исследователи реализуют не только параметрические методы измерения объективной сложности текстов, но и критериальный подход, оценивающий относительную сложность текстов для разных категорий читателей или отдельных лиц (Kupriyanov et al 2023.…”