The TextEvaluator Tool

Towards Understanding Text Factors in Oral Reading

“…Some of these (lexical fre-quency, emotion, arousal) may be associated with local changes in segment and pause durations (see 2.2.2). Many of the features are used as low-level features in readability estimation (Graesser et al, 2004;Sheehan et al, 2014), and are thus likely to capture facets of a reader's experience when reading the text. These included: (1) Vocabulary features capturing presence as well as average score along some meaning dimension, such as concreteness, imageability, emotion, arousal, motion, academic register (Coltheart, 1981;Warriner et al, 2013;Coxhead, 2000); (2) Morphological features (e.g., count of nominalizations, count of syllables); (3) Distributional features such as average word frequency; (4) Syntactic features such as counts of different part-of-speech, as well as features based on specific constructions (relative clauses, preposed clauses etc.…”

An Exploration of Text Difficulty and Knowledge Support on Adolescents' Comprehension

“…Less formal and more conversational language may be easier to comprehend, particularly for readers with low topic familiarity (Beck, McKeown, & Worthy, 1995;Sheehan, Kostin, Napolitano, & Flor, 2014). Formal language includes academic vocabulary, which is defined as both general (e.g., evaluate, synthesize) and discipline-specific terms (e.g., isotope, reconstruction) that appear primarily in academic texts and provide particular challenges to adolescents' reading (Fang & Pace, 2013).…”

“…Such criticisms gave rise to a third generation of measures to estimate text difficulty that include multiple measures of vocabulary, syntax, and the overall discourse structures of text (Benjamin, 2012;Nelson, Perfetti, Liben, & Liben, 2012). Two new measures, Coh-Metrix Text Easability Assessor (Graesser et al, 2011) and TextEvaluator (Sheehan et al, 2014), consider factors that readability measures do not capture, including word familiarity and concreteness, academic language, formality of language, and various types of text cohesion. However, the question remains as to whether these new tools represent differences in multiple text versions of the same content (some deemed easier than others) better than older readability measures.…”

“…Coh-Metrix assesses familiarity based on a database of adult ratings of word familiarity. TextEvaluator considers both the frequency of the word in a text and the relative rarity of a word based on two rare word indexes (Sheehan et al, 2014).…”

“…TextEvaluator uses five measures from the MRC (Medical Research Council) Psycholinguistic database (Coltheart, 1981) to establish a concreteness score. The measures include assessments of the imageability of the words based on various databases of adult ratings of word familiarity or imageability, word frequency in a text, and age of acquisition of the words (Sheehan et al, 2014). Neither Coh-Metrix nor TextEvaluator assesses the relative concreteness or abstractness of the overarching ideas in a text or whether an abstract word or phrase is followed by more concrete examples.…”