Comparing Cognitive Interviewing and Psychometric Methods to Evaluate a Racial/Ethnic Discrimination Scale

Abstract

Proponents of survey evaluation have long advocated the integration of qualitative and quantitative methodologies, but this recommendation has rarely been practiced. We used both methods to evaluate the “Everyday Discrimination” scale (EDS), which measures frequency of various types of discrimination, in a multi-ethnic population. Cognitive testing included 30 participants of various race/ethnic backgrounds and identified items which were redundant, unclear, or inconsistent (e.g., cognitive challenges in quantifying acts of discrimination). Psychometric analysis included secondary data from two national studies, including 570 Asian Americans, 366 Latinos, and 2,884 African Americans, and identified redundant items, as well as those exhibiting differential item functioning (DIF) by race/ethnicity. Overall, qualitative and quantitative techniques complemented one another, as cognitive interviewing findings provided context and explanation for quantitative results. Researchers should consider further how to integrate these methods into instrument pretesting as a way to minimize response bias for ethnic and racial respondents in population-based surveys.

Discrimination and Romani health: a validation study of discrimination scales among Romani women in Macedonia and Serbia

“…Our finding that a reduced-item version of the EDS performed better was similar to previous research in the USA (Chan et al 2012;Kim et al 2014;Reeve et al 2011;Stucky et al 2011). However, two other studies found a single-factor solution using the complete scale, although one of these two studies had deleted the 'courtesy' item a priori due to potential redundancy (Lewis et al 2012;Shariff-Marco et al 2011).…”

“…However, two other studies found a single-factor solution using the complete scale, although one of these two studies had deleted the 'courtesy' item a priori due to potential redundancy (Lewis et al 2012;Shariff-Marco et al 2011). Most studies that have performed DIF analysis on the EDS did so in diverse samples with the primary explanatory variable as race/ethnicity, to determine if the EDS items performed similarly between ethnic groups (Lewis et al 2012;Reeve et al 2011;Shariff-Marco et al 2011). Our approach was similar to Stucky et al which examined DIF of the EDS within African Americans by gender (Stucky et al 2011).…”

Discrimination and Sleep Difficulties during Adolescence: The Mediating Roles of Loneliness and Perceived Stress

“…In our study, we revised this measure to examine adolescents’ attribution for each discriminatory experience they endorsed. In order to assess what participants perceived to be the cause of each discriminatory experience that they reported to have experienced at least once, participants were asked to attribute the experience to one of the following categories: race, gender, age or height/weight (Reeve et al, 2011). The inclusion of multiple discrimination attributions for participants to select from helps to reduce attributional ambiguity, minimize measurement error when examining ethnic/racial discrimination exclusively, and capture other sources of unfair treatment that adolescents encounter (Williams & Mohammed, 2009).…”

Measuring Everyday Racial/Ethnic Discrimination in Health Surveys

“…Items were adapted from the Everyday Discrimination Scale (Williams et al, 1997), with changes in the wording of both specific items and response categories. For instance, the “courtesy” item from the original scale was found to be redundant with “respect” and dropped (Reeve et al, Forthcoming). We added a language item because previous research has found that language discrimination is an important type of racial discrimination faced by ethnic groups (Gee et al, 2009; Spencer and Chen, 2004).…”