Aligning Policy and Methodology to Achieve Consistent Across-Grade Performance Standards

Lewis, Daniel M.; Haug, Carolyn A.

doi:10.1207/s15324818ame1801_2

Cited by 14 publications

(21 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…by establishing cut‐points on the score scale across grade levels through the process of standard‐setting. These proficiency classifications must also be made vertically comparable (Lewis & Haug, 2005). Whether it is possible to establish “vertically moderated standards” (Lissitz & Huynh, 2003) through what is clearly a subjective process is an open question.…”

Section: Discussionmentioning

confidence: 99%

The Impact of Vertical Scaling Decisions on Growth Interpretations

Briggs

Weeks

2009

Educational Measurement

View full text Add to dashboard Cite

Most growth models implicitly assume that test scores have been vertically scaled. What may not be widely appreciated are the different choices that must be made when creating a vertical score scale. In this paper empirical patterns of growth in student achievement are compared as a function of different approaches to creating a vertical scale. Longitudinal item‐level data from a standardized reading test are analyzed for two cohorts of students between Grades 3 and 6 and Grades 4 and 7 for the entire state of Colorado from 2003 to 2006. Eight different vertical scales were established on the basis of choices made for three key variables: Item Response Theory modeling approach, linking approach, and ability estimation approach. It is shown that interpretations of empirical growth patterns appear to depend upon the extent to which a vertical scale has been effectively “stretched” or “compressed” by the psychometric decisions made to establish it. While all of the vertical scales considered show patterns of decelerating growth across grade levels, there is little evidence of scale shrinkage.

show abstract

Section: Discussionmentioning

confidence: 99%

The Impact of Vertical Scaling Decisions on Growth Interpretations

Briggs

Weeks

2009

Educational Measurement

View full text Add to dashboard Cite

show abstract

“…This process is similar to one described by Macy, Bricker, and Squires (2005) in determining the validity and reliability of using a curriculum-based assessment to determine Part C eligibility. This process is also in keeping with recommendations by other researchers, such as Lewis and Haug (2005), who reported that the outcome of setting performance standards would be the generation of a set of cut scores that operationalize a set of performance levels.…”

Section: Validating the Link Between Assessment Items And Child Outcomentioning

confidence: 58%

Kentucky's Early Childhood Continuous Assessment and Accountability System

Rous

McCormick

Gooden

et al. 2007

Topics in Early Childhood Special Education

View full text Add to dashboard Cite

K entucky has developed an assessment and accountability system that provides outcome data for a set of state early childhood standards for children birth to 5 years of age that are aligned with early childhood outcomes designated by the Office of Special Education Programs (OSEP). Within this flexible and comprehensive state assessment system, early intervention and care providers, district preschool personnel, and childcare program staff select a criterion-referenced, curriculum-based assessment (CBA) tool to monitor children's progress. The system is designed so that data from these CBA tools can be used for three purposes: (a) to demonstrate child progress and inform instruction at the individualized intervention or classroom level; (b) to promote program improvement and plan program, district, and state supports through the aggregation of assessment data at the program, district, and state level; and (c) to report on child progress to evaluate the success of federal, state, or district programs and initiatives.

show abstract

“…Several approaches to address the problem have been suggested (Ferrara et al, 2007;Lewis & Haug, 2005). Among them, is incorporating learning progressions in the design of an assessment, such that the vertical articulation takes place explicitly, naturally, and by design, rather than retrospectively (Bejar, Braun, & Tannenbaum, 2007;Kannan, in press).…”

Section: Ecd and Learning Progressionsmentioning

confidence: 99%

Cognitive frameworks for assessment, teaching, and learning: A validity perspective

Kane

Bejar

2014

Psicología Educativa

View full text Add to dashboard Cite

Special Issue:Bringing added value to educational assessment: A shift from an audit mode of assessment to an assistence mode Número especial: Cómo aportar valor añadido a la evaluación: De la auditoría a una función asistencial en la educación Educational PsychologyA Journal for Educational Psychologist 1135-755X/The papers in this issue address the general question of how to add value to educational assessments, particularly in terms of student growth in academic disciplines. In addressing this goal, the papers focus on several recent and emerging model-based methodologies: in particular, learning progressions and cognitive models of learning (Pellegrino, this issue; de la Torre & Minchen, this issue), evidence-centered design (ECD) as a framework for assessment design and development (Zieky, this issue), and cognitively based assessment of, for, and as learning (Deane & Song, this issue; van Rijn et al., this issue).These model-based methodologies involve major developments in how we interpret assessment results and, therefore, they have strong implications for how we evaluate the psychometric quality of the assessments. The model-based interpretations of each student's assessment results involve relatively complex descriptions of each student's achievement emphasizing the student's overall level of sophistication as specified by a list of skills mastered and not mastered (de la Torre & Minchen, this issue), or by a level in a learning progression (Pellegrino, this issue), rather than the student's standing on a unidimensional scale (or on several scales). The goal is to develop assessments that promote learning by providing information that is useful in teaching and learning, and to generate evidence that supports the proposed interpretation and usefulness of the assessment results.Our main point in this paper is that while grounding assessment design in cognitive theories and model-based methodologies is highly desirable, rigorous evaluation of the resulting scores is still necessary. Specifically, the basic definition of validity in terms of the extent to which the interpretation and use of test scores is supported by appropriate evidence and analysis does not need to change. However, as discussed in more detail later, the structure of the arguments used to support the proposed interpretations and uses of the scores and the evidence needed to evaluate these arguments will need to be adapted to fit the proposed interpretations and uses of the test results. Similarly, the analyses of the precision, or reliability, of the results will need to be reconsidered; for example, to the extent that the focus is on placement in a learning progression rather than on a score on a continuous scale, analyses of precision would focus on consistency of placement (in the progression), rather than on traditional reliability or generalizability coefficients. Learning Progressions and Cognitive ModelsCognitive models for learning seek to explain and predict student performance on assessment tasks in terms of profiles of student s...

show abstract

Aligning Policy and Methodology to Achieve Consistent Across-Grade Performance Standards

Cited by 14 publications

References 2 publications

The Impact of Vertical Scaling Decisions on Growth Interpretations

The Impact of Vertical Scaling Decisions on Growth Interpretations

Kentucky's Early Childhood Continuous Assessment and Accountability System

Cognitive frameworks for assessment, teaching, and learning: A validity perspective

Contact Info

Product

Resources

About