Lexical difficulty and diversity of American elementary school reading textbooks

Journal of Educational Psychology

et al. 2018

An adequate level of linguistic complexity in learning materials is believed to be of crucial importance for learning. The implication for school textbooks is that reading complexity should differ systematically between grade levels and between higher and lower tracks in line with what can be called the systematic complexification assumption. However, research has yet to test this hypothesis with a real-world sample of textbooks. In the present study, we used automatic measures from computational linguistic research to analyze 2,928 texts from geography textbooks from four publishers in Germany in terms of their reading demands. We measured a wide range of lexical, syntactic, morphological, and cohesion-related features and developed text classification models for predicting the grade level (Grades 5 to 10) and school track (academic vs. vocational) of the texts using these features. We also tested ten linguistic features that are considered to be particularly important for a reader’s understanding. The results provided only partial support for systematic complexification. The text classification models showed accuracy rates that were clearly above chance but with considerable room for improvement. Furthermore, there were significant differences across grade levels and school tracks for some of the ten linguistic features. Finally, there were marked differences among publishers. The discussion outlines key components for a systematic research program on the causes and consequences of the lack of systematic complexification in reading materials.

Section: Complexity Level In Textbooksmentioning

confidence: 99%

Section: Complexity Level In Textbooksmentioning

confidence: 99%

Reading demands in secondary school: Does the linguistic complexity of textbooks increase with grade level and the academic orientation of the school track?

Berendes

Vajjala

Journal of Educational Psychology

et al. 2018

“…The readability of a text is determined by the combination of all text aspects that affects the reader's understanding, reading speed, and level of interest in the text (Dale and Chall, 1949). Recent studies explore lexical, morphological, semantic, psycholinguistic, syntactic, and cognitive features for determining the reading levels of texts (Crossley et al, 2007;Lu et al, 2014;Hancke et al, 2012;Boston et al, 2008;vor der Brück et al, 2008;Heilman et al, 2007;Feng, 2010;McNamara et al, 2014). Among all these elements, the semantic variable of word difficulty has traditionally been found to account for the greatest percentage of readability variance (Marks et al, 1974).…”

Section: Introductionmentioning

confidence: 99%

“…Recent studies explore lexical, morphological, semantic, psycholinguistic, syntactic, and cognitive features for determining the reading levels of texts (Crossley et al, 2007;Lu et al, 2014;Hancke et al, 2012;Boston et al, 2008;vor der Brück et al, 2008;Heilman et al, 2007;Feng, 2010;McNamara et al, 2014).…”

Section: Introductionmentioning

confidence: 99%

Characterizing Text Difficulty with Word Frequencies

Chen

Proceedings of the 11th Workshop on Innovative Use of NLP For Building Educational Applications

2016

Natural language processing (NLP) methodologies have been widely adopted for readability assessment and greatly enhanced predictive accuracy. In the present study, we study a well-established feature, the frequency of a word in common language use, and systematically explore how such a word-level feature is best used to characterize the reading levels of texts, a text-level classification problem. While traditionally such word-level features are simply averaged for all words of given text, we show that a richer representation leads to significantly better predictive models.A basic approach adding a feature for the standard deviation already shows clear gains, and two more complex options systematically integrating more frequency information are explored: (i) encoding separate means for the words of a text according to which frequency band of the language they occur in, and (ii) encoding the mean of each cluster of words obtained by agglomerative hierarchical clustering of the words in the text based on their frequency. The former organizes frequency around general language characteristics, whereas the latter aims to lose as little information as possible about the distribution of word frequencies in a given text. To investigate the generalizability of the results, we compare cross-validation experiments within a corpus with cross-corpus experiments testing on the Common Core State Standards reference texts. We also contrast two different frequency norms and compare frequency with a measure of contextual diversity. * Xiaobin Chen is also affiliated with the South China University of Technology, where he holds a lecturer position.

“…Later research looked at deeper structural and cognitive variables such as propositional density and coherence for predicting text readability (e.g., Crossley, Greenfield, & McNamara, ; Graesser, McNamara, Louwerse, & Cai, ; Kintsch et al, ; McNamara, Louwerse, McCarthy, & Graesser, ). Recent research has focused on the separate and combined effects of lexical (Crossley, Dufty, McCarthy, & McNamara, ; Flor, Klebanov, & Sheehan, ; Lu, Gamson, & Eckert, ), morphological (François & Watrin, ; Hancke, Vajjala, & Meurers, ), psycholinguistic (Boston, Hale, Kliegl, Patil, & Vasishth, ), semantic (vor der Brück, Hartrumpf, & Helbig, ), syntactic (Heilman, Collins‐Thompson, Callan, & Eskenazi, ), and cognitive (Feng, ; Feng, Elhadad, & Huenerfauth, ; Flor & Klebanov, ; Foltz, Kintsch, & Landauer, ; Graesser, McNamara, & Kulikowich, ; Wolfe et al, ) features on readability by making use of the latest development in Natural Language Processing (NLP) technologies and Machine Learning (ML) methods. Although more and more linguistic and cognitive features have been incorporated into the readability assessment models, it was found that the semantic variable of word difficulty accounts for the greatest percentage of readability variance (Marks, Doctorow, & Wittrock, ).…”

mentioning

confidence: 99%

Word frequency and readability: Predicting the text‐level readability with a lexical‐level attribute

Chen

Journal Research in Reading

2017

Assessment of text readability is important for assigning texts at the appropriate level to readers at different proficiency levels. The present research approached readability assessment from the lexical perspective of word frequencies derived from corpora assumed to reflect typical language experience. Three studies were conducted to test how the word-level feature of word frequency can be aggregated to characterise text-level readability. The results show that an effective use of word frequency for text readability assessment should take a range of characteristics of the distribution of words frequencies into account. For characterizing text readability, taking into account the standard deviation in addition to the mean word frequencies already significantly increases results. The best results are obtained using the mean frequencies of the words in language frequency bands or in bands obtained by agglomerative clustering of the word frequencies in the documentsthough a comparison of within-corpus and cross-corpus results shows the limited generalizability of using high numbers of finegrained frequency bands. Overall, the study advances our understanding of the relationship between word frequency and text readability and provides concrete options for more effectively making use of lexical frequency information in practice. HighlightsWhat is already known about this topic What this paper adds • The paper investigates the impact of frequency norms derived from different corpora on readability assessment in different testing setups. • It compares different types of frequency measures, from occurence counts