Concurrent and Separate Grade-Groups Linking Procedures for Vertical Scaling

Ito, Kyoko; Sykes, Robert C.; Yao, Lihua

doi:10.1080/08957340802161741

Cited by 13 publications

(15 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The debate over the use of separate versus concurrent approaches is relatively new to the research literature, and at present there is no consensus—either theoretically or empirically—as to which should be preferred (Ito, Sykes, & Yao, 2008; Hanson & Béguin, 2002; Kim & Cohen, 1998). In the present study, we develop vertical scales using a separate approach, and a modification of a purely concurrent approach.…”

Section: A Brief Review Of the Literature On Irt‐based Vertical Scalingmentioning

confidence: 99%

The Impact of Vertical Scaling Decisions on Growth Interpretations

Briggs

Weeks

2009

Educational Measurement

View full text Add to dashboard Cite

Most growth models implicitly assume that test scores have been vertically scaled. What may not be widely appreciated are the different choices that must be made when creating a vertical score scale. In this paper empirical patterns of growth in student achievement are compared as a function of different approaches to creating a vertical scale. Longitudinal item‐level data from a standardized reading test are analyzed for two cohorts of students between Grades 3 and 6 and Grades 4 and 7 for the entire state of Colorado from 2003 to 2006. Eight different vertical scales were established on the basis of choices made for three key variables: Item Response Theory modeling approach, linking approach, and ability estimation approach. It is shown that interpretations of empirical growth patterns appear to depend upon the extent to which a vertical scale has been effectively “stretched” or “compressed” by the psychometric decisions made to establish it. While all of the vertical scales considered show patterns of decelerating growth across grade levels, there is little evidence of scale shrinkage.

show abstract

Section: A Brief Review Of the Literature On Irt‐based Vertical Scalingmentioning

confidence: 99%

The Impact of Vertical Scaling Decisions on Growth Interpretations

Briggs

Weeks

2009

Educational Measurement

View full text Add to dashboard Cite

show abstract

“…Against this backdrop and in contrast to previous studies on vertical scaling, in this paper, we suggest validating a new vertical IRT scale and the related scaling decisions by contrasting the calibrated items' empirical difficulties with the items' theoretical content-related difficulties. Thus, we extend previous research on vertical IRT scaling by using an external criterion for validating the vertical scale; specifically, a criterion that allows for justifying the decisions made during test development and item calibration, while considering the concrete, latent construct or ability to be measured, and for verifying the item difficulty parameters and the related growth pattern as the true ones (Harris, 2007;Tong and Kolen, 2007;Ito et al, 2008;Briggs and Weeks, 2009;Dadey and Briggs, 2012). We illustrate this procedure by validating the vertical math scale for formative assessment for third-through ninth-grade students in Northwestern Switzerland by means of cross-sectional data from a pretest calibration sample.…”

Section: Introductionmentioning

confidence: 72%

“…Previous research comparing concurrent and grade-by-grade calibration procedures yielded mixed results and did not provide clear guidance for practical implementation of vertical scales based on IRT methods. Some studies identified concurrent calibration as superior to grade-by-grade calibration (e.g., Kim and Cohen, 1998;Hanson and Béguin, 1999), whereas others reported the opposite (e.g., Béguin et al, 2000;Ito et al, 2008). In addition, several studies point out diverse interactions between the calibration procedure and various other decisions during the development of the vertical scale, such as the choice of IRT model, data collection design, number of linking items, or test specifications (Hanson and Béguin, 2002;Pomplun et al, 2004;Tong and Kolen, 2007;Briggs and Weeks, 2009;Lei and Zhao, 2012; see also Harris, 2007).…”

Section: Vertical Scaling Based On Irt Methodsmentioning

confidence: 99%

Development and Validation of a Vertical Scale for Formative Assessment in Mathematics

et al. 2019

View full text Add to dashboard Cite

The regular formative assessment of students' abilities across multiple school grades requires a reliable and valid vertical scale. A vertical scale is a precondition not only for comparing assessment results and measuring progress over time, but also for identifying the most informative items for each individual student within a large item bank independent of the student's grade to increase measurement efficiency. However, the practical implementation of a vertical scale is psychometrically challenging. Several extant studies point to the complex interactions between the practical context in which the scale is used and the scaling decisions that researchers need to make during the development of a vertical scale. As a consequence, clear general recommendations are missing for most scaling decisions. In this study, we described the development of a vertical scale for the formative assessment of third-through ninth-grade students' mathematics abilities based on item response theory methods. We evaluated the content-related validity of this new vertical scale by contrasting the calibration procedure's empirical outcomes (i.e., the item difficulty estimates) with the theoretical, content-related item difficulties reflected by the underlying competence levels of the curriculum, which served as a content framework for developing the scale. Besides analyzing the general match between empirical and content-related item difficulty, we also explored, by means of correlation and multiple regression analyses, whether the match differed for items related to different curriculum cycles (i.e., primary vs. secondary school), domains, or competencies within mathematics. The results showed strong correlations between the empirical and content-related item difficulties, which emphasized the scale's content-related validity. Further analysis showed a higher correlation between empirical and content-related item difficulty at the primary compared with the secondary school level. Across the different curriculum domains and most of the curriculum competencies, we found comparable correlations, implying that the scale is a good indicator of the math ability stated in the curriculum.

show abstract

“…Before the linking process, the quality of the common items is checked by examining the linear relationship of the parameter estimates for the common items. (Ito, Sykes, & Yao, 2008;Karkee, Lewis, Hoskens, Kim, 2007;Meng, Kolen, & Lohman, 2006;Tong & Kolen, 2007), and at the same time, produce differences on the scale score (Chin, Kim, & Nering, 2006;Ito et al, 2008;Kim, 2007;2003;Tong & Kolen, 2007).…”

Section: Calibration Methodsmentioning

confidence: 99%

Investigating the effects of missing data treatments on item response theory vertical scaling

Shin¹

View full text Add to dashboard Cite

years in graduate school. Thanks to Dr. Deidre Egan in the writing center for reading and editing a number of pages and helping me to improve the language in this work. I would also like to thank my Smiling Puppy team members and other colleagues in DC for celebrating my defense. I enjoy working with them and I have learned a lot from them about educational testing. I would also like to thank my parents, YoungJune Shin and EunSoo Lee, and parents inlaw, GunSeop Chang and SeokYoung Hong, for their prayers, sacrifice, and endless love. I feel blessed that I have such spiritual and loving parents. I also thank my lovely sisters, SunAe and JeeWon, who are best friends I can trust and rely on. Also thank to my wonderful sisters in-law, SooJung and KeeSook, for their prayers and encouragement. My little one, Yoon, has motivated me greatly to finish this process. His smile and snuggle make me feel so happy every moment. He has given me an enormous gift that I would never have understood without him. I promise we will spend more time together from now on. I also express my most heartfelt gratitude to my husband Kee. I would never have finished this work without his unending support, love and care. The second part of my life has been blessed and happy because of him, and I am excited to continue our lives together. Lastly, I would like to thank God, who accompanies me and provides me with the best that I could ever imagine. Whenever I felt frustrated during this process, he encouraged me to stand again by making me to look what is beyond and to acknowledge the true fulfillment I should pursue in my life.

show abstract

Concurrent and Separate Grade-Groups Linking Procedures for Vertical Scaling

Cited by 13 publications

References 10 publications

The Impact of Vertical Scaling Decisions on Growth Interpretations

The Impact of Vertical Scaling Decisions on Growth Interpretations

Development and Validation of a Vertical Scale for Formative Assessment in Mathematics

Investigating the effects of missing data treatments on item response theory vertical scaling

Contact Info

Product

Resources

About