Acquiescence and Person Differential Functioning: Solving Person DIF with Balanced Scales

Abstract

Likert-type self-report scales are frequently used in large-scale educational assessment of social-emotional skills. Self-report scales rely on the assumption that their items elicit information only about the trait they are supposed to measure. However, different response biases may threat this assumption. Specifically in children, the response style of acquiescence is an important source of systematic error. Balanced scales have been proposed as a solution to control for acquiescence, but the reasons why this design feature worked from the perspective of modern psychometric models have been underexplored. Three methods for controlling for acquiescence are compared: (a) Classical ipsatization by partialling out the mean, (b) an Item Response Theory method to measure Person Differential Functioning (DPF), and (c) random intercept item factor analysis. Comparative analyses are conducted on simulated ratings and on self-ratings provided by 40,649 students (aged 11 to 18) on a fully balanced 30-item scale assessing Conscientious Self-management. Acquiescence bias was explained as person differential item functioning and it was demonstrated that: (a) the acquiescence index is highly related to DPF, (b) balanced scales resolve DPF, and (c) that random intercept factors are highly related to DPF and the substantive factor is controlled for DPF.