Differential Weighting: A Review of Methods and Empirical Studies

Wang, Marilyn W.; Stanley, Julian C.

doi:10.2307/1169462

Cited by 50 publications

(77 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This strategy is sample dependent and has been shown to have no advantage in tests containing more than 10 items. 10,11 In differential option weighting, weights are assigned to alternative answers. The weights assigned to alternative answers can be empirically based on previous or present test administrations, as seen with differential item weighting.…”

Section: ■ Introductionmentioning

confidence: 99%

Polytomous versus Dichotomous Scoring on Multiple-Choice Examinations: Development of a Rubric for Rating Partial Credit

et al. 2013

View full text Add to dashboard Cite

The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes vital that instructors reach general agreement as to the level of partial credit. Using workshops with instructors, ACS Exams has identified reasons why partial credit could be assigned in an exam set of 70 test items. With partial-credit assignments thus established, polytomous scoring is applied and the effect of such scoring on the overall norm-referenced psychometrics is determined and described. While individual students move within the overall norm, the average influence of polytomous scoring as conceived by the workshops does not substantially change the ability to do norm-based comparisons. ABSTRACT: The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes vital that instructors reach general agreement as to the level of partial credit. Using workshops with instructors, ACS Exams has identified reasons why partial credit could be assigned in an exam set of 70 test items. With partial-credit assignments thus established, polytomous scoring is applied and the effect of such scoring on the overall norm-referenced psychometrics is determined and described. While individual students move within the overall norm, the average influence of polytomous scoring as conceived by the workshops does not substantially change the ability to do norm-based comparisons.

show abstract

Section: ■ Introductionmentioning

confidence: 99%

Polytomous versus Dichotomous Scoring on Multiple-Choice Examinations: Development of a Rubric for Rating Partial Credit

et al. 2013

View full text Add to dashboard Cite

show abstract

“…Several other weights could have been chosen. For example, a variety of strategies for weighting item scores and item responses are reviewed by Wang and Stanley (1970). However, the purpose of item weighting in the studies reviewed by Wang and Stanley was to achieve gains in reliability or validity; the purpose of item weighting in the present study was to characterize the effect of test speededness on test performance.…”

Section: Ni Is the Number Of Items In The Testmentioning

confidence: 93%

The Effect of Test Speededness on Subgroup Performance

Lawrence¹

1993

ETS Research Report Series

View full text Add to dashboard Cite

The purpose of this study was to gauge the disadvantage, if any, of reaching difficult items that appear earlier in a test, at the expense of not reaching easier items that appear later in a test. The study focused on SAT and GMAT data and examined the effect for the following focal groups: females, Asian American, Blacks, and Hispanics. The impact of differential test speededness on subgroup differences in test scores was found to be minimal for SAT Mathematical and GMAT Quantitative tests. However, there appears to be a relationship between the presence of differential speededness on the SAT Verbal test and subgroup differences in test performance; after matching on total score, Black and Hispanic examinees receive credit for more difficult sets of items in the SAT Verbal test, relative to White examinees. Due to smaller sample sizes, findings for the GMAT Verbal tests were not as clear-cut as the findings for the SAT Verbal tests. This research attempted to provide a technique for assessing the circumstances under which the effect of test speededness has a differential effect on the performance of SUbgroups, when differences in ability are taken into consideration. As is the case with all non-experimental research on test speededness, the results of this study cannot predict how examinees would have performed with a shorter test, or more liberal time limits, because test strategies might have been different under those conditions. THE EFFECT OF TEST SPEEDEDNESS ON SUBGROUP PERFORMANCEAs part of an effort to evaluate test speededness, one area of concern is the degree to which subgroup performance is differentially affected by not completing a test section. A difference in scores between subgroups may occur when candidates in particular SUbgroups tend not to reach the relatively easy items appearing at the end of a test section. The purpose of this study is to gauge the disadvantage, if any, of reaching the difficult items that appear earlier in the test, at the expense of not reaching the easier items that appear later in the test.When a test is speeded, it is desirable to order items within a test according to their difficulty, with the easier items appearing at the beginning of the test and the harder items appearing at the end of the test. Under this kind of item ordering, students who are unable to complete the test will at least have an opportunity to attempt those items for which the probability of a correct response is highest (assuming that students pace through the items in the order presented). For this reason, a typical recommendation for test construction is that test items in sections be arranged in ascending order of difficulty (e.g., Cronbach, 1951).Sometimes it is not feasible to order items from easy to hard. An example of a test in which the items cannot all be ordered from easy to hard is a test that contains passage-based reading comprehension items. In general, reading questions do not progress from easy to difficult. Instead, the questions follow the logic and organization of the...

show abstract

“…disaggregating the patents into fewer industries, tends to decreases the correlation between ratio and efficiency measure. This is related to the general tendency of weighting schemes to lose their importance when the number of weighted items increases (Wang & Stanley, 1970).…”

Section: Figurementioning

confidence: 99%

The innovation efficiency of German regions – a shared-input DEA approach

Broekel¹,

Rogge

Brenner

2017

Rev Reg Res

View full text Add to dashboard Cite

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Institute of Economic and Cultural Geography, Leibnitz University, Hannover. Terms of use: Documents in Nicky RoggeFaculty of Business and Economics Naamsestraat 69, Katholieke Universiteit Leuven. Thomas BrennerEconomic Geography and Location Research, Philipps-University, Marburg. Abstract:The paper contributes to the debate on how to measure regions' innovation performance. On the basis of the concept of regional innovation efficiency, we propose a new measure that eases the issue of choosing between industry-specific or global measures. We argue for the use of a robust shared-input DEA-model to estimate regions' innovation efficiency in a global manner, while it can be disaggregated into industry-specific innovation efficiency measures. The latter is particularly useful when relating the innovative output to the R&D input involves the use of blurry matching procedures.We illustrate the use of the method by investigating the innovation efficiency as well as its change in time of German labor market regions. It is shown that the method treats regions that have industry structures skewed towards industries with high and low innovation intensities more fairly than traditional approaches.

show abstract

Differential Weighting: A Review of Methods and Empirical Studies

Cited by 50 publications

References 0 publications

Polytomous versus Dichotomous Scoring on Multiple-Choice Examinations: Development of a Rubric for Rating Partial Credit

Polytomous versus Dichotomous Scoring on Multiple-Choice Examinations: Development of a Rubric for Rating Partial Credit

The Effect of Test Speededness on Subgroup Performance

The innovation efficiency of German regions – a shared-input DEA approach

Contact Info

Product

Resources

About