Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features

Lee, Bruce W.; Jang, Yoo Sung; Lee, Jason Hyung-Jong

doi:10.18653/v1/2021.emnlp-main.834

Cited by 30 publications

(47 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It would also be interesting to assess the effects of other domain-general psychological factors and emotions in L2 acquisition and learning. Another possible area of future research would be to examine the text readability thoroughly by using machine learning models ( Lee et al, 2021 ). The current study particularly adopted Flesch-Kincaid Grade Level in the correlation, regression and SEM analysis.…”

Section: Discussionmentioning

confidence: 99%

The Impact of Academic Procrastination on Second Language Writing: The Mediating Role of L2 Writing Anxiety

Zhang

2022

Front. Psychol.

View full text Add to dashboard Cite

Recently, there has been a surge of interest in the exploration of psychological properties in a second language context. Considerable literature has grown up around the influence of these psychological properties on L2 writing specifically. However, the impact of academic procrastination, which is an important psychological property, has been understudied and it remains unclear how affective factors in L2 might play a role in the above potential influence on L2 writing. Therefore, the current study explored the impact of academic procrastination on L2 writing and examined the mediating role of L2 writing anxiety, by adopting text readability as an innovative approach to assessing L2 writing performance. Participants were 55 Chinese speakers of L2 English. By utilizing the collected questionnaire data and the readability indicators of the L2 writing task, the current research conducted correlation analysis, regression analysis, and structural equation modeling analysis. The results revealed that academic procrastination had a significant negative impact on the readability indicator of Flesch-Kincaid Grade Level in L2 writing. L2 writing anxiety played a complete mediating role in the impact. Academic procrastination can significantly affect Flesch-Kincaid Grade Level of L2 writing indirectly through L2 writing anxiety. Pedagogical implications and future studies were discussed.

show abstract

Section: Discussionmentioning

confidence: 99%

The Impact of Academic Procrastination on Second Language Writing: The Mediating Role of L2 Writing Anxiety

Zhang

2022

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…We used the Welch t-test (Welch, 1947) with Bonferroni correction resulting to an alpha level of 0.0167 (0.05/3). For the variables of interest, we extracted over 160 linguistic features using the LingFeat tool by Lee et al (2021)…”

Section: Difference In Prompt and Continuation Complexitiesmentioning

confidence: 99%

“…In this study, we formally introduce the task of uniform complexity for text generation or UCTG in an open-ended narrative generation setting using the WRITINGPROMPTS dataset from Reddit (Fan et al, 2018). To cover a wide range of measures used for approximating linguistic complexity, we extracted over 160 features drawing concepts from age-of-acquisition in developmental studies, word and phrase level part-of-speech, discourse from entity mentions, and formula-based readability indices (Lee et al, 2021). We compare text continuations from three models: humans, an off-the-shelf GPT-model, and a finetuned GPT-2 model.…”

Section: Introductionmentioning

confidence: 99%

Uniform Complexity for Text Generation

Imperial¹

2022

Preprint

View full text Add to dashboard Cite

Powerful language models such as GPT-2 have shown promising results in tasks such as narrative generation which can be useful in an educational setup. These models, however, should be consistent with the linguistic properties of triggers used. For example, if the reading level of an input text prompt is appropriate for lowleveled learners (ex. A2 in the CEFR), then the generated continuation should also assume this particular level. Thus, we propose the task of uniform complexity for text generation which serves as a call to make existing language generators uniformly complex with respect to prompts used. Our study surveyed over 160 linguistic properties for evaluating text complexity and found out that both humans and GPT-2 models struggle in preserving the complexity of prompts in a narrative generation setting.

show abstract

“…Several attempts have been made to measure readability, i.e. (a) eye movements, (b) word difficulty, (c) semantic richness (Lee et al 2021), (d) N-gram analysis (Xia et al 2016) and (e) cognitivemotivated features (Feng et al 2009). Readability has been measured through a variety of metrics such as word length, sentence length.…”

Section: Previous Studiesmentioning

confidence: 99%

Text genres, readability and readers’ comprehensibility

Li¹

2022

EJCSIT

View full text Add to dashboard Cite

This study applies mathematical linguistics to explore the association between text genres, readability and readers’ comprehensibility. The target readers are students from Korea, Thailand and China, studying in Japanese universities, pursuing 20 different disciplines. Japanese is their third language, and all students have passed the Japanese-Language Proficiency Test Level 1. The textbook’s readability and readers’ comprehensibility are measured at two levels using two metrics. Mean dependency distance (MDD) is employed for measuring syntactic diversity; moving-average morphological richness (MAMR) and moving-average mean size of paradigm (MAMSP) are calculated for measuring lexicon diversity. The findings indicate that in terms of lexical diversity, textbooks of humanities seem simpler than natural science and engineering. In syntactic complexity, the textbook of informatics shows the simplest structure while that of social welfare presents the highest. Text genres relate to a textbook’s readability and eventually influence readers’ comprehensibility. Moreover, lexical diversity is not corelated to syntactic complexity.

show abstract

Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features

Cited by 30 publications

References 65 publications

The Impact of Academic Procrastination on Second Language Writing: The Mediating Role of L2 Writing Anxiety

The Impact of Academic Procrastination on Second Language Writing: The Mediating Role of L2 Writing Anxiety

Uniform Complexity for Text Generation

Text genres, readability and readers’ comprehensibility

Contact Info

Product

Resources

About