Effects of Population Heterogeneity on Accuracy of DIF Detection

Oliveri, María Elena; Ercikan, Kadriye; Zumbo, Bruno D.

doi:10.1080/08957347.2014.944305

Cited by 23 publications

(14 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Test fairness review processes typically involve analyzing items by investigating differential item functioning (DIF) posttest administration. Ercikan and Oliveri (2013) and Oliveri et al (2014) identified shortcomings related to DIF analyses when tests are administered to heterogeneous populations, including under detecting DIF, suggesting the need to expand review processes from analyzing DIF post-administration to considering fairness starting from the conceptualization and design of assessments.…”

Section: Introductionmentioning

confidence: 99%

Considerations for Designing Accessible Educational Scenario-Based Assessments for Multiple Populations: A Focus on Linguistic Complexity

Oliveri

2019

Front. Educ.

View full text Add to dashboard Cite

As the diversity of the test-taker population increases so should assessment development practices evolve to consider the various needs of the multiple populations taking the assessments. One need is the ability to understand the language used in test items and tasks so they do not present unnecessary challenges for the test-takers, which may be misconstrued as a lack of knowledge of the content assessed. This investigation is important because linguistic complexity may constitute a source of construct-irrelevant variance, which may render an assessments' passages and questions less accessible and present unnecessary challenges to the multiple test-taker populations potentially leading to score misinterpretation, disengagement with the task, and increased cognitive load. To develop more linguistically accessible assessments for multiple populations, less accessible construct-irrelevant text may require modification and less accessible construct-relevant text may need scaffolding. In this paper, I discuss considerations for designing accessible assessments for multiple populations with a focus on linguistic complexity. To illustrate these considerations, I refer to digitally delivered scenario-based tasks of English Language Arts framed in a science context.

show abstract

Section: Introductionmentioning

confidence: 99%

Considerations for Designing Accessible Educational Scenario-Based Assessments for Multiple Populations: A Focus on Linguistic Complexity

Oliveri

2019

Front. Educ.

View full text Add to dashboard Cite

show abstract

“…Em síntese, pelas evidências apresentadas, verificaram-se evidências satisfatórias para o instrumento frente ao modelo Rasch empregado, o que possibilitou a análise de funcionamento diferencial dos itens (DIF). Destaca-se que a inspeção prévia de adequação dos dados ao modelo Rasch minimiza a possibilidade de apurar diferenças em função de erros assistemáticos, o que torna as inspeções realizadas a priori indicadas ao se buscar DIF entre grupos (Oliveri, Ercikan, & Zumbo, 2014). Em relação ao funcionamento diferencial, analisaram-se as diferenças em função de três critérios, que foram empregados anteriormente na literatura: 1) diferenças entre grupos com valores t superiores a t> 2.00) diferenças no parâmetro de dificuldade b DIF >|.40| entre os grupos; e 3) nível mínimo de significância estatística para as diferenças encontradas igual a p<.05 (Ambiel, Carvalho, Moreira, & Bacan, 2016;Linacre, 2009;Primi, Carvalho, Miguel, & Silva, 2010).…”

Section: Tabela 1 Análise De Dimensionalidade E Resíduos Por Componenunclassified

Intención emprendedora en la universidad

Cortêz

Veiga

2019

CienciasPsi

View full text Add to dashboard Cite

La actividad emprendedora puede ser una alternativa ocupacional para estudiantes universitarios, especialmente en un escenario con alto desempleo. En el presente estudio, buscamos verificar el funcionamiento diferencial (DIF) de una escala de intención emprendedora (EIQBr-b) en estudiantes de humanidades (n= 50), ciencias sociales (n= 50), salud (n= 47) y ciencias exactas (n= 50). Utilizamos Rasch-Rating-Scale-Score Model y analizamos el DIF mediante el método de Mantel-Haenszel para ítems politómicos. Encontramos que los estudiantes de ciencias sociales presentaban menos dificultades en los ítems que enfatizaban la proposición planificada de su propia compañía. Los ítems relacionados con la autodeterminación y la carrera profesional no presentaron DIF entre las diferentes áreas. Por eso, es necesario planificar, desarrollar y mejorar las acciones de educación y orientación profesional para los emprendedores, considerando las particularidades entre las áreas de conocimiento que diferencian la manifestación de la intención emprendedora entre los grupos.

show abstract

“…Whitmore and Schumacker (1999) conducted a simulation with DIF size variations of 5% and 15%. The latest research by increasing the proportion of smaller DIF sizes is 0%, 15%, and 30% (Oliveri, Ercikan, & Zumbo, 2014).…”

Section: Differential Item Functioningmentioning

confidence: 99%

“…Wingen3.0 is a response data generation program that can be conditioned based on the needs of research analysis. The generation of data using the Wingen3.0 program was also carried out by (Han, 2007;Oliveri et al, 2014). The response data generation settings are explained in the next procedure.…”

Section: Data Generationmentioning

confidence: 99%

Sensitivity and Accuracy of the Mantel-Haenszel Method and Standardization Method: Detection of Item Functioning Differential

Rustam

Naga

Supriyati

2019

IJELS

View full text Add to dashboard Cite

Detection of differential item functioning (DIF) is needed in the development of tests to obtain useful items. The Mantel-Haenszel method and standardization are tools for DIF detection based on classical theory assumptions. The study was conducted to highlight the sensitivity and accuracy between the Mantel-Haenszel method and the standardization method in DIF detection. Simulation design (a) test participants consisted of 1000 responses in both the reference and focus groups, (b) the size of the proportion of DIF (0.1; 0.25; 0.50; and 0.75), and (c) the length of the multiple choice test with 40 choices the answer. Research shows that the Mantel-Haenszel method has the same sensitivity as the standardization method in DIF proportions of 10% and 25%, however, when the ratio of DIF proportions above 25% the standardization method is less sensitive, and conversely the sensitivity of the Mantel-Haenszel method increases. The standardization method has higher accuracy than the Mantel-Haenszel method in the DIF proportion of 10%, however, when the size of the DIF proportion above 10% the accuracy of the standardization method decreases, the accuracy of the Mantel-Haenszel method is higher than the standardization method. Thus, if the ratio of DIF is detected by the standardization method of (≤0.10), then the results of the standardization method are preferred as a reference. Conversely, if the proportion of DIF detected by the standardization method is (≥0.10), then the result of the Mantel-Haenszel method is chosen as a reference.

show abstract

Effects of Population Heterogeneity on Accuracy of DIF Detection

Cited by 23 publications

References 27 publications

Considerations for Designing Accessible Educational Scenario-Based Assessments for Multiple Populations: A Focus on Linguistic Complexity

Considerations for Designing Accessible Educational Scenario-Based Assessments for Multiple Populations: A Focus on Linguistic Complexity

Intención emprendedora en la universidad

Sensitivity and Accuracy of the Mantel-Haenszel Method and Standardization Method: Detection of Item Functioning Differential

Contact Info

Product

Resources

About