A Graphical Approach to Evaluating Equating Using Test Characteristic Curves

Abstract

An essential concern in the application of any equating procedure is determining whether tests can be considered equated after the tests have been placed onto a common scale. This article clarifies one equating criterion, the first-order equity property of equating, and develops a new method for evaluating equating that is linked to this criterion. The new approach involves graphically examining the difference in test characteristic curves (TCCs), calculating the maximum absolute difference between the TCCs, and comparing the differences in TCCs to the difference that matters (DTM). The new approach is applied to evaluate the equating of the Multistate Bar Exam (MBE) for six different item response theory (IRT) scaling approaches in the common item nonequivalent groups design. The new approach is also contrasted with Tong and Kolen’s (2005) index for assessing first-order equity. The empirical investigations indicated that the Stocking-Lord and fixed-parameter equating methods appear to perform the best for equating the MBE and that the use of concurrent calibration is not desirable.

Normalization procedure for the Baptista Depression Scale - Adult Version (EBADEP-A): transferring of norms

“…With the porpoise to analyze the instruments on the same scale, it was necessary to calibrate and equalize the measures of test parameters, so that the Theta values could be used to the transfer the cutoff point from the CES-D to EBADEP-A (Bauer & Hussong, 2009;Embretson & Reise, 2000;Smith et al, 2006;Thomas, 2011;Wyse & Reckase, 2011). After the procedures of equalization and anchorage, all possible CES-D scores (from zero to 60) were associated with an ability level, from which we could…”

“…Furthermore, measurements of both instruments are also calibrated on a common scale, allowing the comparison and the transfer of normative standards from one to another, through the equalization procedure (Bauer & Hussong, 2009;Embretson & Reise, 2000;Thomas, 2011). The equalization process allows instruments that theoretically measure the same construct, to become comparable and proportional, making it possible that the same statistical significance (and so their interpretations) to be attributed to participants with the same ability (Smith et al, 2006;Wyse & Reckase, 2011).…”

A Comparison of IRT Vertical Scaling Methods in Determining the Increase in Science Achievement

“…In addition, the fact that eighth grade students have entered adolescence may have affected their psychology and achievements negatively. In the study of Briggs, Weeks and Wiley (2009), parallel to this finding, it was stated that the growth patterns did not show an increase from one year to the (Tong, & Kolen, 2008;Cetin, 2009;Wysel, & Reckase, 2011;Altun, 2013). In Tong and Kolen (2010)'s study, it was found that the mean difference was higher in the lower class levels and the mean difference decreased as the class level increased.…”

Two Statistics for Measuring the Score Comparability of Computerized Adaptive Tests

“…Lord (1980) showed that in the strictest sense it was impossible for these conditional distributions to be equal unless the alternate forms were strictly parallel or perfectly reliable. Neither situation happens in practice, so weaker forms of Lord's equity property are often used instead (see Kolen & Tong, 2005;Wyse & Reckase, 2011). Two weaker forms of equity are first-order equity and second-order equity.…”

“…For example, Tong and Kolen (2005) proposed two indices, including one based on first-order equity and one based on second-order equity, and applied these indices to compare equipercentile, IRT observed score, and IRT true score equating methods. Similarly, Wyse and Reckase (2011) suggested a first-order equity index based on comparing TCCs and used this method to compare six IRT equating methods for the Multistate Bar Exam. Simple comparisons of curves for CATs are difficult because there are thousands of forms, item selection for each form is tailored based on ability, and there is not a single base form to which all forms are equated.…”