Improved Type I Error Control and Reduced Estimation Bias for DIF Detection Using SIBTEST

Jiang, Hai; Stout, William

doi:10.3102/10769986023004291

Cited by 48 publications

(41 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are a number of methods available for detecting uniform DIF, including the Mantel-Haenszel test (MH), logistic regression (LR), and SIBTEST (Millsap & Everson, 1993;Shealy & Stout, 1993). Several studies have demonstrated the effectiveness of these methods under a wide variety of simulated conditions (e.g., Rogers & Swaminathan, 1993;Roussos & Stout, 1996;Jiang & Stout, 1998;Narayanan & Swaminathan, 1994). While collectively this work identified some minor differences in the effectiveness of these DIF detection methods, generally speaking they were found to perform similarly well in most situations.…”

mentioning

confidence: 65%

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

Finch

2011

Applied Measurement in Education

View full text Add to dashboard Cite

Methods of uniform differential item functioning (DIF) detection have been extensively studied in the complete data case. However, less work has been done examining the performance of these methods when missing item responses are present. Research that has been done in this regard appears to indicate that treating missing item responses as incorrect can lead to inflated Type I error rates (false detection of DIF). The current study builds on this prior research by investigating the utility of multiple imputation methods for missing item responses, in conjunction with standard DIF detection techniques. Results of the study support the use of multiple imputation for dealing with missing item responses. The article concludes with a discussion of these results for multiple imputation in conjunction with other research findings supporting its use in the context of item parameter estimation with missing data.Missing data are a common problem in many data analysis contexts, including those associated with item response data. For example, respondents may simply overlook one or more items on an instrument and neglect to provide a response. In other cases, individuals may not respond to an item for some systematic reason, such as being unsure of the answer to a test question. A number of methods Correspondence should be addressed to Holmes Finch,

show abstract

mentioning

confidence: 65%

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

Finch

2011

Applied Measurement in Education

View full text Add to dashboard Cite

show abstract

“…Some researchers have found that SIBTEST and Rasch methods are comparable in terms of identifying DIF items and in terms of type 1 error (Narayanan & Swaminathan, 1994;Rogers & Swaminathan, 1993;Roussos & Stout, 1996). Others have found SIBTEST to be more effective in identifying DIF items (e.g., Bolt, 2002;Jiang & Stout, 1998).…”

Section: Research Questionsmentioning

confidence: 95%

Gender DIF in Reading and Mathematics Tests With Mixed Item Formats

Taylor

Lee

2012

Applied Measurement in Education

View full text Add to dashboard Cite

This was a study of differential item functioning (DIF) for grades 4, 7, and 10 reading and mathematics items from state criterion-referenced tests. The tests were composed of multiple-choice and constructed-response items. Gender DIF was investigated using POLYSIBTEST and a Rasch procedure. The Rasch procedure flagged more items for DIF than did the simultaneous item bias procedure-particularly multiple-choice items. For both reading and mathematics tests, multiple-choice items generally favored males while constructed-response items generally favored females. Content analyses showed that flagged reading items typically measured text interpretations or implied meanings; males tended to benefit from items that asked them to identify reasonable interpretations and analyses of informational text. Most items that favored females asked students to make their own interpretations and analyses, of both literary and informational text, supported by text-based evidence. Content analysis of mathematics items showed that items favoring males measured geometry, probability, and algebra. Mathematics items favoring females measured statistical interpretations, multistep problem solving, and mathematical reasoning.Differential item functioning (DIF) analyses are a routine part of large-scale assessment programs. The goal of these studies is to ensure that test items do not unfairly assist or impair individual students' performances because of their race, ethnicity, or gender. In addition, items identified during bias and sensitivity reviews are not necessarily the ones flagged with statistical methods (Englehard,

show abstract

“…We used a nonparametric approach to the assessment of DIF as implemented in the SIBTEST statistical software package (for a technical description of the SIBTEST procedure, see Sheely andStout, 1992a,b andLiang andStout, 1998). SIBTEST estimates respondents' depression levels by counting the number of questions with which respondents agree.…”

Section: Detecting Difmentioning

confidence: 99%

Differential Item Functioning Between Ethnic Groups in the Epidemiological Assessment of Depression

Breslau¹,

Javaras²,

Blacker³

et al. 2008

Journal of Nervous &Amp; Mental Disease

View full text Add to dashboard Cite

A potential explanation for the finding that disadvantaged minority status is associated with a lower lifetime risk for depression is that individuals from minority ethnic groups may be less likely to endorse survey questions about depression even when they have the same level of depression. We examine this possibility using a nonparametric item response theory approach to assess differential item functioning (DIF) in a national survey of psychiatric disorders, the National Comorbidity Survey. Of 20 questions used to assess depression symptoms, we found evidence of DIF in 3 questions when comparing non-Hispanic blacks with non-Hispanic whites and in 3 questions when comparing Hispanics with non-Hispanic whites. However, removal of the questions with DIF did not alter the relative prevalence of depression between ethnic groups. Ethnic differences do exist in response to questions concerning depression, but these differences do not account for the finding of relatively low prevalence of depression among minority groups. KeywordsDepression; measurement; bias; ethnicity; epidemiology Although risk of depression is consistently associated with low socioeconomic status (Lorant et al., 2003), epidemiological studies have not found elevated risk for depression among socially disadvantaged minority groups in the United States. In fact, all surveys that have assessed depression in national samples of American adults have found significantly lower lifetime risk for depression among non-Hispanic blacks compared with non-Hispanic whites (Blazer et al., 1994;Breslau et al., 2006;Somervell et al., 1989;Weissman et al., 1991;Williams et al., 2007). In addition, no surveys have found higher risk for depression among Hispanics relative to non-Hispanic whites, whereas some have found significantly lower risk among Hispanics than non-Hispanic whites (Blazer et al., 1994;Breslau et al., 2006;Karno et al., 1987). However, because of the difficulty of establishing the cultural equivalence of NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript psychological constructs (Butcher et al., 2003;Janca, 2005;Room et al., 1996), including depression (Kleinman, 2004), concerns about the validity of this counterintuitive finding remain (Rogler, 1999). Measurement bias resulting from differential misclassification, with minorities being less likely to meet DSM criteria despite similar levels of underlying disorder, remains a potential methodological explanation for these findings (Rogler et al., 2001).In this article, we investigate whether measurement bias in the assessment of depression can account for the unexpectedly low lifetime prevalence of depression among Hispanics and nonHispanic blacks relative to non-Hispanic whites. Data come from the National Comorbidity Survey (NCS), a nationally representative survey of psychiatric disorders in which respondents were interviewed face-to-face by nonclinician interviewers using the Composite International Diagnostic Interview (CIDI), a fully structured diagnostic inter...

show abstract

Improved Type I Error Control and Reduced Estimation Bias for DIF Detection Using SIBTEST

Cited by 48 publications

References 12 publications

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

Gender DIF in Reading and Mathematics Tests With Mixed Item Formats

Differential Item Functioning Between Ethnic Groups in the Epidemiological Assessment of Depression

Contact Info

Product

Resources

About