Posterior Predictive Model Checking for Multidimensionality in Item Response Theory

Levy, Roy; Mislevy, Robert J.; Sinharay, Sandip

doi:10.1177/0146621608329504

Cited by 76 publications

(156 citation statements)

References 26 publications

(58 reference statements)

Supporting

Mentioning

145

Contrasting

Order By: Relevance

“…It follows that the adequacy of a given IRT model can be tested by determining whether the off-diagonal elements of the residual covariance matrix tend to vanish (Stout, 1990;McDonald & Mok, 1995). A Bayesian model-checking method along these lines has been given by Levy, Mislevy, and Sinharay (2009). It also follows that the RMSR and GFI carry over as global fit indices from the factor analysis of tests to the IRT analysis of items.…”

Section: Nonlinear Factor Analysis Of Binary Items and Item Response mentioning

confidence: 99%

A unified factor-analytic approach to the detection of item and test bias: Illustration with the effect of providing calculators to students with dyscalculia

Lee¹,

Lee²,

Wells³

et al. 2016

TQMP

View full text Add to dashboard Cite

An absence of measurement bias against distinct groups is a prerequisite for the use of a given psychological instrument in scientific research or high-stakes assessment. Factor analysis is the framework explicitly adopted for the identification of such bias when the instrument consists of a multi-test battery, whereas item response theory is employed when the focus narrows to a single test composed of discrete items. Item response theory can be treated as a mild nonlinearization of the standard factor model, and thus the essential unity of bias detection at the two levels merits greater recognition. Here we illustrate the benefits of a unified approach with a real-data example, which comes from a statewide test of mathematics achievement where examinees diagnosed with dyscalculia were accommodated with calculators. We found that items that can be solved by explicit arithmetical computation became easier for the accommodated examinees, but the quantitative magnitude of this differential item functioning (measurement bias) was small.

show abstract

Section: Nonlinear Factor Analysis Of Binary Items and Item Response mentioning

confidence: 99%

A unified factor-analytic approach to the detection of item and test bias: Illustration with the effect of providing calculators to students with dyscalculia

Lee¹,

Lee²,

Wells³

et al. 2016

TQMP

View full text Add to dashboard Cite

show abstract

“…The pairwise diagnostic tools that have been proposed to detect LID in item response data have traditionally been studied and applied in non-adaptive settings, with both dichotomous and polytomous response data (e.g., Yen, , 1993Chen & Thissen, 1997;Ip, 2001;Kim, De Ayala, Ferdous, & Nering, 2007;Levy, Mislevy, & Sinharay, 2009;Zenisky, Hambleton, & Sireci, 2002). However, the data resulting from an adaptive instrument differs from data from a fixed-form instrument in two notable ways .…”

Section: Detecting Local Item Dependencementioning

confidence: 99%

“…assume that there is a separate trait that is common to each set of locally dependent items but is not common to the rest of the items on the instrument (Chen & Thissen, 1997;Levy, Mislevy, & Sinharay, 2009;Thissen, Bender, Chen, Hayashi, & Wiesen, 1992). In other words, all items have a non-zero weight, or "loading" on the common trait.…”

Section: Underlying Local Dependence Underlying Local Dependence (Ulmentioning

confidence: 99%

“…Popular LID statistics have been applied to dichotomous non-adaptive data (Chen & Thissen, 1997;Kim et al, 2007;Levy, Mislevy, & Sinharay, 2009;, polytomous non-adaptive data , and dichotomous adaptive data . Findings of these studies with regards to the Q 3 and X 2 statistic are reviewed below and summarized in Table 4.…”

Section: Examining Properties Of Lid Statisticsmentioning

confidence: 99%

“…have conducted as few as 10 or 50 replications (Levy, Mislevy, & Sinharay, 2009; and as many as 1,000 replications per condition (Kim et al, 2007). However, 100 replications were frequently selected by researchers interested in developing an empirical distribution for the LID statistic and 95% critical values under the null condition (Chen & Thissen, 1997;Chen & Wang, 2007;; 100 replications also allows for intuitive calculations of power and false positive rates for other conditions using the cut-off obtained under the null condition.…”

Section: Replications Previous Studies Investigating the Performancementioning

confidence: 99%

See 2 more Smart Citations

Detecting Local Item Dependence in Polytomous Adaptive Data

Mislevy

Rupp

Harring

2012

J Educational Measurement

View full text Add to dashboard Cite

iii Table 6. Results summary for tracked item pairs in the CONV condition with no LID .. 91 1 For simplicity, the term "individual" will be used throughout the document to denote the person who provides the responses to a data-collection instrument; the term is assumed to be synonymous with alternative terms such as "examinee" and "respondent," though it is acknowledged that each has domainspecific connotations. Along these lines, the data-collection instrument will be referred to as an "instrument," (representing alternative terms such as "test" and "questionnaire") and the term "trait" is used to denote the construct measured by the items (representing alternative terms such as "ability" and "proficiency"). Exceptions are made in rare instances when necessary to remain consistent with the literature. 2This initiative focuses on more accurate and efficient measurement of patient-reported symptoms and aspects of health-related quality of life. A primary goal of PROMIS is to develop instruments based on IRT methodologies in these domains that are publically available for the clinical research community.In IRT, estimates of respondents' traits (θ) are based not only on the responses they provide, but also the characteristics (i.e., parameters) of the items they are administered such as their difficulty -reflected by category boundary parameters (b) -their ability to differentiate among respondents -reflected by slope parameters (a) -and their susceptibility to guessing -reflected by lower asymptote parameters (c).One unidimensional IRT model frequently applied in health-outcomes settings is the graded response model (GRM;Samejima, 1969). The GRM is appropriate for item responses that fall in multiple ordered categories. It predicts the conditional probability of an individual responding in a particular category as a function of an individual's latent trait value and several item parameters.The GRM is considered a "difference model" (Thissen & Steinberg, 1986) or "indirect IRT model" (Embretson & Reise, 2000) because the probabilities are computed in two stages. Following Dodd, De Ayala, and Koch (1995), the probability that individual i will produce a response in category k or higher for item j is first computed as:where θ i is the individual's trait level, a j is the discrimination parameter for item j, and b jk is the category boundary for category k for item j. In the second step, the actual category 3 response probabilities for each response are computed by subtracting the cumulative probabilities from adjacent response categories conditional on θ using:There are a total of K -1 boundary parameters that need to be estimated for an item with K score categories. Figure 1 illustrates the probability of choosing each of the response options offered with an item from PROMIS designed to measure fatigue impact and experience according to the GRM. It shows that a person with a low level of fatigue would have a high probability of indicating that his fatigue made it "not at all hard to carry on a conversation." In ...

show abstract

Item Response Theory

Carlson

Davier

2013

ETS Research Report Series

View full text Add to dashboard Cite

Few would doubt that ETS researchers have contributed more to the general topic of item response theory (IRT) than individuals from any other institution. In this report, we briefly review most of those contributions, dividing them into sections by decades of publication, beginning with early work by Fred Lord and Bert Green in the 1950s and ending with recent work that produced models involving complex structures and multiple dimensions.

show abstract

Posterior Predictive Model Checking for Multidimensionality in Item Response Theory

Cited by 76 publications

References 26 publications

A unified factor-analytic approach to the detection of item and test bias: Illustration with the effect of providing calculators to students with dyscalculia

A unified factor-analytic approach to the detection of item and test bias: Illustration with the effect of providing calculators to students with dyscalculia

Detecting Local Item Dependence in Polytomous Adaptive Data

Item Response Theory

Contact Info

Product

Resources

About