Abstract:Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS staff to ensure that they meet established scientific and professiona… Show more
“…The component skills part of the assessment targeted generalizable features related to basic processing of text (e.g., word recognition), while the integrated skills part aimed to capture the ability to accomplish performance on purpose‐driven, thematically related, integrated tasks, called global‐integrated, scenario‐based assessments (GISAs). O'Reilly and Sabatini () described how several dimensions of the reading ability construct could be organized and measured through scenarios. They state: This orchestration is achieved by incorporating a scenario‐based design that organizes the assessment around a central theme and goal for reading (e.g., work with fellow students to study for an exam or prepare a presentation on a science or history topic) (.…”
Section: Selected Areas Of Current Assessment Activitymentioning
This article summarizes some of the main issues, concerns, and debates that have ensued over the years in the field of L2 assessment and shows how past concerns have shaped contemporary L2 assessment research and practice. The article first describes what L2 assessment is and what it entails, arguing that notions of L2 assessment have been broadened over the years to keep pace with contemporary interpretations of how information from assessments in large‐scale or classroom contexts is used to make decisions. It then describes in some detail four approaches to construct definition, showing how these approaches have influenced what gets assessed. Finally, the paper discusses a range of other topics of current and future importance for L2 assessment theory, research, and practice.
“…The component skills part of the assessment targeted generalizable features related to basic processing of text (e.g., word recognition), while the integrated skills part aimed to capture the ability to accomplish performance on purpose‐driven, thematically related, integrated tasks, called global‐integrated, scenario‐based assessments (GISAs). O'Reilly and Sabatini () described how several dimensions of the reading ability construct could be organized and measured through scenarios. They state: This orchestration is achieved by incorporating a scenario‐based design that organizes the assessment around a central theme and goal for reading (e.g., work with fellow students to study for an exam or prepare a presentation on a science or history topic) (.…”
Section: Selected Areas Of Current Assessment Activitymentioning
This article summarizes some of the main issues, concerns, and debates that have ensued over the years in the field of L2 assessment and shows how past concerns have shaped contemporary L2 assessment research and practice. The article first describes what L2 assessment is and what it entails, arguing that notions of L2 assessment have been broadened over the years to keep pace with contemporary interpretations of how information from assessments in large‐scale or classroom contexts is used to make decisions. It then describes in some detail four approaches to construct definition, showing how these approaches have influenced what gets assessed. Finally, the paper discusses a range of other topics of current and future importance for L2 assessment theory, research, and practice.
“…The second type of assessment design—global, integrated scenario‐based assessment (GISA)—is intended to measure a broader conception of reading literacy ability (see Bennett, ; Deane et al., ; O'Reilly, Sabatini, Bruce, Pillarisetti, & McCormick, ; O'Reilly & Sabatini, in press; O'Reilly & Sheehan, ; Sheehan & O'Reilly, ). The theoretical foundations of scenario‐based assessments are meant to be consistent with the more general features of a number of cognitive models and theories of comprehension.…”
Section: Two Kinds Of Assessments For Two Kinds Of Learnersmentioning
In recent years, researchers, educators, and policy makers have called for a new generation of reading comprehension assessments (e.g., Partnership for 21st Century Skills, 2008). Advocates of this movement argue for a deeper type of reading assessment, one that captures students’ ability to not only understand single texts in isolation but also to engage in purposeful, multisource integration of sources. While this shift in how we define and measure reading comprehension is laudable, assessments must also measure the fundamental reading skills that may impede higher‐level comprehension processes. This article presents data from two assessments that were designed to work in tandem to provide a more complete picture of reading comprehension. Middle school students were given a component skills battery which measured core reading skills such as word recognition, decoding, vocabulary, and morphology, as well a second assessment designed to measure reading comprehension. Reading comprehension was measured using a scenario‐based assessment approach, which required students to read a range of sources to fulfill a particular reading goal. The results indicate that students, including struggling readers, were able to read, understand, and problem solve in complex learning environments, but students’ ability to do so was often tempered by their basic reading skills. We argue that including a measure of component skills alongside a measure of higher‐level comprehension is beneficial in interpreting student performance. Accordingly, we present the results on the scenario‐based measure as a function of reading component skills and argue for the value of using this approach for struggling readers.
“…The resulting assessment resembles a performance task (because it provides a context and purpose for all the specific tasks that it contains), but it has many of the advantages of a traditional standardized test because it measures targeted skills through a series of separate items. CBAL researchers have been developing the case for SBAs for some time, both within ELA (O'Reilly & Sabatini, ; O'Reilly & Sheehan, ; Sheehan & O'Reilly, ) and in other content domains (e.g., Graf, Harris, Marquez, Fife, & Redman, ; Liu, Rogat, & Bertling, ).…”
mentioning
confidence: 99%
“…Then, we discuss how these key practices connect with the existing CBAL framework and with related frameworks, such as Reading for Understanding (RFU; cf. O'Reilly & Sabatini, ; Sabatini et al, ). Next, we examine how an analysis of key practices can inform the structure and sequence of tasks in CBAL SBAs.…”
This paper presents a framework intended to link the following assessment development concepts into a systematic framework: evidence‐centered design (ECD), scenario‐based assessment (SBA), and assessment of, for, and as learning. The context within which we develop this framework is the English language arts (ELA) for K‐12 students, though the framework could easily be applied to cover reading, writing, and critical thinking skills from pre‐K through college. Central to the framework is the concept of a key practice, drawn from constructivist learning theory, which emphasizes the purposeful social context within which skills are recruited and organized to carry out complex literacy tasks. We argue that key practices provide a key link between existing CBAL™ ELA learning progressions (defined as part of a student model for literacy skills) and the structure of well‐designed SBAs. This structure enables us to design assessments that model a key practice, supporting the systematic creation of task sequences that can be used to support both instruction and assessment.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
