Linguistic Features of Writing Quality

Abstract

In this study, a corpus of expert-graded essays, based on a standardized scoring rubric, is computationally evaluated so as to distinguish the differences between those essays that were rated as high and those rated as low. The automated tool, Coh-Metrix, is used to examine the degree to which high- and low-proficiency essays can be predicted by linguistic indices of cohesion (i.e., coreference and connectives), syntactic complexity (e.g., number of words before the main verb, sentence structure overlap), the diversity of words used by the writer, and characteristics of words (e.g., frequency, concreteness, imagability). The three most predictive indices of essay quality in this study were syntactic complexity (as measured by number of words before the main verb), lexical diversity (as measured by the Measure of Textual Lexical Diversity), and word frequency (as measured by Celex, logarithm for all words). Using 26 validated indices of cohesion from Coh-Metrix, none showed differences between high- and low-proficiency essays and no indices of cohesion correlated with essay ratings. These results indicate that the textual features that characterize good student writing are not aligned with those features that facilitate reading comprehension. Rather, essays judged to be of higher quality were more likely to contain linguistic features associated with text difficulty and sophisticated language.

MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment

“…Various studies have shown MTLD to be at least as effective as the industry standard vocd-D index, and even one of the most informative and distinguishing variables in the entire arsenal of several hundred Coh-Metrix indices (see Crossley, Salsbury, & McNamara, 2009;McNamara et al, 2010). Such successes do not mean that all the settings of the MTLD approach are optimal.…”

“…This default factor size is used in the MTLD prototype that features in the Coh-Metrix suite of textual analysis indices and is the factor size used in the engineering studies that have incorporated MTLD (e.g., Crossley & McNamara, in press;McNamara et al, 2010).…”

Cohesive Errors in Writing among ESL Pre-Service Teachers

“…Hence, it is also relevant and important to investigate these aspects in teachers' writing as an indication of their writing proficiency and quality. While there have been studies done on students' cohesion and use of cohesive devices in writing (Crossley & McNamara, 2010a;McNamara, Crossley, & McCarthy, 2010;Tangkiengsirisin, 2010), little research has been done on cohesive errors (Sadighi & Heydari, 2012). However, Sadighi and Heydari's (2012) study did find that students of differing levels of proficiency made more errors in different types of cohesion devices.…”

Preface: Special Issue on Multidisciplinary Approaches to AI and Education for Reading and Writing

“…They test three Coh-Metrix indices that had been found sufficient to predict human-scored writing quality of average-performing college students (McNamara et al 2010). While these indices are not good predictors for the low-performing adults, ten other Coh-Metrix indices were identified that were predictive for this population.…”