Measuring Reading Strategy Use

Abstract

This study describes the development and validation of a multidimensional measure of preadolescent and adolescent readers' abilities to apply reading comprehension strategies necessary for understanding challenging academic texts. The Strategy Use Measure (SUM) was designed with the intention of being pedagogically informative to the increasingly multilingual student population in the U.S. in grades 6 through 8. The SUM aims to measure four areas of knowledge and skill that are widely purported to support the use of reading strategies: (a) morphological awareness, (b) knowledge of cognates, (c) ability to relate micro-and macro-ideas within a text, and (d) the ability to use intra-and inter-sentential context clues for defining unfamiliar words. The test was developed following a principled, iterative process to instrument development, employing Rasch models and qualitative investigations to test hypotheses related to the instrument's validity. Findings suggest promising evidence for the validity and fairness of this multidimensional measure.

Raising Critical Readers in the 21st Century: A Case of Assessing Fourth-Grade Reading Abilities and Practices

“…The long-standing tensions about the what and the how of reading comprehension development and ways to assess it stems (at least in part) from a lack of consensus on the theoretical underpinnings of such development (Perfetti & Stafura, 2014). Assessments of reading comprehension are hence a reflection of this lack of consensus (Arya et al, 2020b;Valencia & Pearson, 1986) and as such, warrant closer inspection of the ways that reading assessments are used for making school-based decisions. In California, districts have the freedom to choose reading assessments for informing programmatic and instructional decisions.…”

“…Further, our development of this progression of texts followed an iterative effort of (a) conducting think-aloud sessions with young children from our aforementioned six-member panel of young students and (b) debriefing and revising drafted texts according to multiple dimensions of textual complexity as well as sociocultural relevance (Arya & Maul, 2021;Arya et al, 2020b). As suggested earlier, words and concepts included in each leveled text of the CRA aligned with readability and coherence information that allowed for comparisons with STAR test scores (see Online Appendix 1).…”

“…Recorded sessions and notes for each assessed student (over 15 h of audio footage and approximately 50 pages of notes summarized) were analyzed by three separate researchers that included an undergraduate research assistant, a graduate student with expertise in literacy instruction, and a literacy scholar with strong expertise in literacy assessment and text development. Three analysts separately reviewed each recorded session to determine the depth of understanding and metacognitive engagement based on previous research on comprehension assessment (Arya & Maul, 2012;Arya et al, 2020b) and the use of Wilson's (2004) building blocks framework for building transparent maps of variations in metacognitive engagement. Such mapping efforts (i.e., clarifying a progression or depth of understanding and reasoning) are not to suggest that our constructs of interest (comprehension of textual media and metacognitive engagement) are linear in nature; given the complexities involved in reading and the ways that we synthesize and integrate new information, there are arguably countless maps that could be drawn to trace variations of understanding and perspective-taking.…”