Bayesian Identification of Outliers in Computerized Adaptive Tests

Bradlow, Eric T.; Weiss, Robert E.; Cho, Meehyung

doi:10.2307/2669829

Cited by 20 publications

(39 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that the method proposed by van is based on the order of presentation. Bradlow et al (1998) note that using the order of presentation warm-up outliers can be detected. Those are examinees who have trouble settling in or warming up to the exam due to unfamiliarity or nervousness.…”

Section: Discussionmentioning

confidence: 99%

“…To detect item score patterns with many incorrect answers at the end of the test (due to, for example, fatigue) the item order can be reversed and the same methodology can be applied. Also choosing a , 5 k 5 a2 is possible (Bradlow et al, 1998). A limitation of this method is that to set these boundaries additional knowledge should be available.…”

Section: Discussionmentioning

confidence: 99%

“…As an alternative to both methods discussed, Bradlow, Weiss, and Cho (1998) c; = c,= 0, (10) where C' and Creflect the sum of consecutive positive and negative residuals, respectively. Let UB and LB be some appropriate upper and lower bounds.…”

Section: Easy[pi(e)mentioning

confidence: 99%

“…A distribution of the statistics is needed to decide if an item score pattern is unlikely under the model. In CAT there are different possibilities for selecting the distribution f(x) (Bradlow et al, 1998). The main question is on what information should be "conditioned?…”

Section: Sampling Distributionmentioning

confidence: 99%

See 3 more Smart Citations

Outlier Detection in High‐Stakes Certification Testing

Meijer

2002

J Educational Measurement

View full text Add to dashboard Cite

Recent developrnents of person‐Jit analysis in computerized adaptive testing (CAT) are discussed. Methods from stutistical process control are presented that have been proposed to classify an item score pattern as fitting or misjitting the underlying item response theory model in CAT. Most person‐fit research in CAT is restricted to simulated data. In this study, empirical data from a certification test were used, Alternatives are discussed to generate norms so that bounds can be determined to classify an item score pattern as fitting or misfitting. Using bounds determined from a sample of a high‐stakes certification test, the empirical analysis showed that dizerent types of misfit can be distinguished. Further applications using statistical process control methods to detect misfitting item score patterns are discussed.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Easy[pi(e)mentioning

confidence: 99%

Section: Sampling Distributionmentioning

confidence: 99%

See 2 more Smart Citations

Outlier Detection in High‐Stakes Certification Testing

Meijer

2002

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…The decision to categorize sets of observations such as item-score vectors as either fitting or misfitting is known as outlier identification (cf., Barnett & Lewis, 1994, p. 7). Recent examples can be found in certification testing (Meijer, 2002) and adaptive testing (Bradlow & Weiss, 2001;Bradlow, Weiss, & Cho, 1998). In person-fit analysis, the interest is mainly with identifying aberrant item-score vectors and inferring the cause of this aberrance, for example, for diagnostic reasons (e.g., did the respondent understand the test instruction?…”

mentioning

confidence: 99%

Outlier Detection in Test and Questionnaire Data

Zijlstra

Ark

Sijtsma

2007

Multivariate Behavioral Research

View full text Add to dashboard Cite

Classical methods for detecting outliers deal with continuous variables. These methods are not readily applicable to categorical data, such as incorrect/correct scores (0/1) and ordered rating scale scores (e.g., 0; : : : ; 4) typical of multi-item tests and questionnaires. This study proposes two definitions of outlier scores suited for categorical data. One definition combines information on outliers from scores on all the items in the test, and the other definition combines information from all pairs of item scores. For a particular item-score vector, an outlier score expresses the degree to which the item-score vector is unusual. For ten real-data sets, the distribution of each of the two outlier scores is inspected by means of Tukey's fences and the extreme studentized deviate procedure. It is investigated whether the outliers that are identified are influential with respect to the statistical analysis performed on these data. Recommendations are given for outlier identification and accommodation in test and questionnaire data.

show abstract

Residual‐based cumulative sum charts to monitor time series of counts via copula‐based Markov models

Alqawba

Kim

Radwan

2022

Appl Stoch Models Bus & Ind

View full text Add to dashboard Cite

Several scientific observations produce data that consist of serially dependent counts that are difficult to accurately analyze due to the absence of normality and the limited literature on dealing with such data. In this article, we propose a cumulative sum chart to monitor serially dependent counts using copula-based Markov models. After reviewing such models, we introduce the randomized quantile residuals obtained from the Markov process. The proposed method is evaluated using a comprehensive simulation study and a real-life example.

show abstract

Bayesian Identification of Outliers in Computerized Adaptive Tests

Cited by 20 publications

References 12 publications

Outlier Detection in High‐Stakes Certification Testing

Outlier Detection in High‐Stakes Certification Testing

Outlier Detection in Test and Questionnaire Data

Residual‐based cumulative sum charts to monitor time series of counts via copula‐based Markov models

Contact Info

Product

Resources

About