Differential Item Functioning (DIF) in composite health measurement scale: Recommendations for characterizing DIF with meaningful consequences within the Rasch model framework

Abstract

Objective The aims were to review practices concerning Differential Item Functioning (DIF) detection in composite measurement scales, particularly those used in health research, and to provide guidance on how to proceed if statistically significant DIF is detected. Methods This work specifically addressed the Rasch model which is the subject of growing interest in the field of health owing to its particularly advantageous properties. There were three steps: 1) Literature review to describe current practices; 2) Simulation study to determine under which conditions encountered in health research studies can erroneous conclusions be drawn from group comparisons when a scale is affected by DIF but which is not considered; 3) Based on steps 1 and 2, formulation of recommendations that were subsequently reviewed by leading internationally recognized experts. Results Four key recommendations were formulated to help researchers to determine whether statistically significant DIF is meaningful in practice, according to the kind of DIF (uniform or non-uniform) and the DIF effect size. Conclusion This work provides the first recommendations on how to deal in practice with the presence of DIF in composite measurement scales used in health research studies.

The Factorial Validity of the Norwegian Version of the Multicomponent Training Distress Scale (MTDS-N)

“…However, further investigations are recommended to produce a more nuanced picture of the presence of DIF in the MTDS-N. If the scale is to be modified, different authors have proposed solutions to handle the presence of DIF in practice [ 118 ]. According to the authors of the review, researchers have recommended to split items exhibiting DIF to calibrate them in each group separately when the scale is used in a study; to remove items exhibiting DIF from the scale; or reformulate items exhibiting DIF [ 118 ].…”

The WHOQOL-BREF: a modern psychometric evaluation of its internal construct validity in people with multiple sclerosis

“…Where a testlet or super-item solution is obtained, and DIF is shown to be still present, the substantive nature of this DIF is tested by comparing unadjusted and adjusted person estimates. Should the t test of this comparison (for paired or repeated measures) be significant, then an effect size of the difference is calculated which should be less than 0.1, in which case DIF is deemed to be small and no action is taken [26]. All analyses were performed using RUMM2030 software [27].…”

Validity of the French version of the Autonomy Preference Index and its adaptation for patients with advanced cancer

“…� I would rather the decision be left to me, after having taken my doctor's advice into consideration 24 (13) � I would rather decide together with my doctor 71 (38) � I would prefer to let my doctor decide, once my opinion has been taken into consideration 56 (30) � I would prefer to let my doctor decide alone 30 (16) � Missing 1 (0)…”

“…According to Mokkink et al and Milsap, "a measuring device should function in the same way across varied conditions, so long as those varied conditions are irrelevant to the attribute being measured" [27,28]. We studied measurement invariance across age, sex and education level as usually performed and recommended, across French and English languages to ensure the comparability of the scores from both language versions, and across both settings (primary care patients and patients with incurable cancer) to check the likeness of the API factor structure in these settings [28][29][30]. The Consensus-based Standards for the selection of health Measurement Instruments (COS-MIN) guidelines were followed to report the results [31].…”