Appropriateness measurement with polychotomous item response models and standardized indices

Abstract

The test scores of some examinees on a multiple choice test may not provide satisfactory measures of their abilities. The goal of appropriateness measurement is to identify such individuals. Earlier theoretical and expcrimcntal work considered examinees answering all, or almost all, test items. This article reports research that extends appropriateness measurement methods to examinees with moderately high non-response ratcs. These methods treat non-response as if it were a deliberate option choice and then attempt to measure the 'appropriateness' of the pattern of option choices. Earlier studies used only the dichotomous pattern of 'right' and 'not right' answers. A general polychotomous model is introduced along with a technique called 'standardization' designed to reduce the observed confounding between measured appropriateness and ability. A standardized appropriateness index based on a polychotomous model yielded higher rates of detection of simulated spuriously low examinees than the analogous index based on a dichotomous model. However, the converse was true for simulated spuriously high examinees. Standardization was found to reduce greatly the interaction between ability and measured appropriateness.

Pratiques de pondération des résultats d’enquêtes à grande échelle aux notes scolaires en mathématiques

“…Baumert et demmrich (2001) définissent la motivation à passer une épreuve comme étant « la volonté de s'impliquer à répondre aux items de l'épreuve et d'investir effort et persistance à l'activité entreprise » (traduction libre, p. 441). la revue de la littérature révèle que l'approche de l'élève vis-à-vis des enquêtes à grande échelle influence significativement ses résultats (drasgow, levine, & williams, 1985 ;Glas & Falcón, 2003 ;Glas & Meijer, 2003 ;levine & rubin, 1979 ;nering, 1997 ;van Krimpen-stoop & Meijer, 1999). un corpus grandissant de recherches décrit le phénomène à travers lequel l'élève ne mobilise pas toutes ses ressources et ne fournit pas nécessairement l'effort attendu pour remplir le test lorsque ce dernier n'a pas ou peu de conséquence personnelle (Baumer & demmrich, 2001 ;Blau, Moller, & Jones, 2004 ;deMars, 2000 ;eklöf, 2010 ;Ferrer-caja & weiss, 2002 ;hoyt, 2001 ;olaru, Karageorghis, & chatzisarantis, 2003 ;sloane & Kelly, 2003 ;van Barneveld et al, 2012 ;wise, 1996a, 1996bwise & demars, 2005 ;wolf & smith, 1995 ;wolf, smith, & Birnbaum, 1995).…”

Using SAS PROC NLMIXED to fit item response theory models

“…In this section, we will briefly explain the popular residual fit statistics (Wright & Masters, 1982) of the Rasch model, implemented in the software WINSTEPS (Linacre & Wright, 1999). Discussions of other diagnostic indices can be found in Drasgow, Levine, and Williams (1985). The starter data example is used to illustrate the computations of these fit statistics as implemented by PROC IML (Interactive Matrix Language) using output from PROC NLMIXED.…”

An investigation of enhancement of ability evaluation by using a nested logit model for multiple-choice items

“…In person-fit area, Wollack (1997), Drasgow, Levine, and Williams (1985) were used distractor information to detect aberrant response behaviors. Kim (2006) found that linking procedure was improved when distractor information was added in.…”