Providing Subscale Scores for Diagnostic Information: A Case Study When the Test is Essentially Unidimensional

Stone, Clement A.; Ye, Feifei; Zhu, Xiaowen; Lane, Suzanne

doi:10.1080/08957340903423651

Cited by 30 publications

(43 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Table 1 includes the results for the Spring 2006 assessment of the Delaware Student Testing Program (DSTP) 8th grade mathematics assessment. The data from the test were analyzed in Stone, Ye, Zhu, & Lane (2010), who reported, using an exploratory factor analysis method, the presence of only one factor in the data set. The four subscores that were proposed, but are not reported, correspond to four content domains: numeric reasoning, algebraic reasoning, geometric reasoning, and quantitative reasoning.…”

Section: Review Of Results From Operational Data Analysismentioning

confidence: 99%

How Often Do Subscores Have Added Value? Results from Operational and Simulated Data

Sinharay

2010

J Educational Measurement

106

153

View full text Add to dashboard Cite

Recently, there has been an increasing level of interest in subscores for their potential diagnostic value. Haberman suggested a method based on classical test theory to determine whether subscores have added value over total scores. In this article I first provide a rich collection of results regarding when subscores were found to have added value for several operational data sets. Following that I provide results from a detailed simulation study that examines what properties subscores should possess in order to have added value. The results indicate that subscores have to satisfy strict standards of reliability and correlation to have added value. A weighted average of the subscore and the total score was found to have added value more often.

show abstract

Section: Review Of Results From Operational Data Analysismentioning

confidence: 99%

How Often Do Subscores Have Added Value? Results from Operational and Simulated Data

Sinharay

2010

J Educational Measurement

106

153

View full text Add to dashboard Cite

show abstract

“…The seventh row of Stone et al (2009), who reported, using the exploratory factor analysis method, the presence of only one factor in the data set. The four subscores, which were proposed but are not reported, correspond to four content domains: numeric reasoning, algebraic reasoning, geometric reasoning, and quantitative reasoning.…”

Section: Review Of Results From Operational Data Analysismentioning

confidence: 99%

“…The next two tests were discussed in Harris and Hanson (1991). The next, DSTP Math, is discussed in Stone, Ye, Zhu, and Lane (2009). The next eight, denoted CA-CH, are certification tests discussed in Puhan et al (2008).…”

Section: Review Of Results From Operational Data Analysismentioning

confidence: 99%

“…The four subscores, which were proposed but are not reported, correspond to four content domains: numeric reasoning, algebraic reasoning, geometric reasoning, and quantitative reasoning. The summary statistics reported in Stone et al (2009) were used to compute the PRMSEs required in the method of Haberman (2008). 5 For some tests such as CC, average disattenuated correlation was larger than 1-they were set to 1.00.…”

Section: Review Of Results From Operational Data Analysismentioning

confidence: 99%

See 1 more Smart Citation

When Can Subscores Be Expected to Have Added Value? Results From Operational and Simulated Data

Sinharay

2010

ETS Research Report Series

View full text Add to dashboard Cite

Recently, there has been an increasing level of interest in using subscores for their potential diagnostic value. Haberman (2008) suggested a method based on classical test theory to determine whether subscores have added value over total scores. This paper provides a literature review and reports when subscores were found to have added value for several operational data sets.Then this paper provides results from a detailed simulation study that examines what properties subscores should possess in order to have added value. The results indicate that subscores have to satisfy strict standards of reliability and correlation to have added value. Augmented subscores (Haberman, 2008;Wainer et al., 2001) were found to have added value more often.

show abstract

“…These conditions were met in the present application, in which reliabilities of the subtests were at least .85 and twodimensional models fitted substantially better than unidimensional models. Moreover, in cases were there is essentially one dominant factor or highly correlated dimensions, MIRT modeling has been shown to yield subscale scores that have improved reliability over unadjusted subscale scores, because the correlational structure is taken into account (De la Torre & Patz, 2005, Stone, Ye, Zhu & Lane, 2010.…”

Section: Practical Implicationsmentioning

confidence: 98%

The Language Factor in Elementary Mathematics Assessments: Computational Skills and Applied Problem Solving in a Multidimensional IRT Framework

Hickendorff¹

2013

Applied Measurement in Education

View full text Add to dashboard Cite

The results of an exploratory study into measurement of elementary mathematics ability are presented. The focus is on the abilities involved in solving standard computation problems on the one hand and problems presented in a realistic context on the other. The objectives were to assess to what extent these abilities are shared or distinct, and the extent to which students' language level plays a differential role in these abilities. Data from a sample of over 2,000 students from first, second, and third grade in the Netherlands were analyzed in a multidimensional item response theory (IRT) framework. The latent correlation between the two ability dimensions (computational skills and applied mathematics problem solving) ranged from .81 in grade 1 to .87 in grade 3, indicating that the ability dimensions are highly correlated but still distinct. Moreover, students' language level had differential effects on the two mathematical abilities: Effects were larger on applied problem solving than on computational skills. The implications of these findings for measurement practices in the field of elementary mathematics are discussed.

show abstract

Providing Subscale Scores for Diagnostic Information: A Case Study When the Test is Essentially Unidimensional

Cited by 30 publications

References 13 publications

How Often Do Subscores Have Added Value? Results from Operational and Simulated Data

How Often Do Subscores Have Added Value? Results from Operational and Simulated Data

When Can Subscores Be Expected to Have Added Value? Results From Operational and Simulated Data

The Language Factor in Elementary Mathematics Assessments: Computational Skills and Applied Problem Solving in a Multidimensional IRT Framework

Contact Info

Product

Resources

About