Adjusting for Subgroup Differences in Extreme Response Tendency in Ratings of Health Care: Impact on Disparity Estimates

Abstract

Objective. Adjust for subgroup differences in extreme response tendency (ERT) in ratings of health care, which otherwise obscure disparities in patient experience. Data Source. 117,102 respondents to the 2004 Consumer Assessment of Healthcare Providers and Systems (CAHPS) Medicare Fee-for-Service survey. Study Design. Multinomial logistic regression is used to model respondents' use of extremes of the 0-10 CAHPS rating scales as a function of education. A new two-stage model adjusts for both standard case-mix effects and ERT. Ratings of subgroups are compared after these adjustments. Principal Findings. Medicare beneficiaries with greater educational attainment are less likely to use both extremes of the 0-10 rating scale than those with less attainment. Adjustments from the two-stage model may differ substantially from standard adjustments and resolve or attenuate several counterintuitive findings in subgroup comparisons. Conclusions. Addressing ERT may be important when estimating disparities or comparing providers if patient populations differ markedly in educational attainment. Failures to do so may result in misdirected resources for reducing disparities and inaccurate assessment of some providers. Depending upon the application, ERT may be addressed by the two-stage approach developed here or through specified categorical or stratified reporting.

Less Use of Extreme Response Options by Asians to Standardized Care Scenarios May Explain Some Racial/Ethnic Differences in CAHPS Scores

“…Other studies have shown that non-English-preferring Asians tend to exhibit even lower ERT than English-preferring Asians. 3,5,18 These findings, along with those from our study, suggest that direct case-mix adjustment for Asian race/ethnicity, or indirect adjustment such as Asian language spoken at home (recently adopted for HCAHPS) 43 or Asian survey language (as used for MA/PDP CAHPS), 25 may improve the measurement of the quality of care provided by physicians, medical groups, hospitals, and health plans. Such adjustments would have little effect for most providers, but would be likely to notably and correctly increase the scores of those with large proportions of Asian patients.…”

“…13 Some studies suggest that compared with whites, 4,9,14-16 Latinos and perhaps African Americans are more likely to use responses at the extreme ends of the scale, a measurement properly known as Extreme Response Tendency (ERT). [17][18][19] In contrast, Asians show less ERT than whites, [20][21][22][23] and lower ERT may explain why Asians report worse experiences with care than whites. Because of the skewness of CAHPS scores, in which most ratings fall in the most positive categories (eg, 9 or 10 on a 0-10 rating scale), 24 avoidance of the extremes (low ERT) by Asians could result in lower mean scores overall, as avoiding positive extremes lowers the mean, and the negative extreme is rare enough to have little consequence.…”

“…Because of the skewness of CAHPS scores, in which most ratings fall in the most positive categories (eg, 9 or 10 on a 0-10 rating scale), 24 avoidance of the extremes (low ERT) by Asians could result in lower mean scores overall, as avoiding positive extremes lowers the mean, and the negative extreme is rare enough to have little consequence. 18,25 Much of the extant evidence on ERT comes from observational data. To more confidently assess whether certain patient groups use experience with care response scales differently, the care being rated needs to be held constant.…”

Measuring Patient Experience: Concepts and Methods

“…A further difficulty in interpreting patient experience data is that different population groups may place greater emphasis on some aspects of care than others-for example, less educated patients may place greater relative value on continuity of care than more educated patients [34]. An example of different ways in which different population groups might interpret the actual response options presented on a questionnaire was reported by Elliott et al [35], who found that more educated patients were less likely to use extreme response options at both ends of a scale. If these differences were pronounced, it would make it more difficult to draw comparisons between population groups across a range of quality domains.…”

Measuring Patient Safety in Primary Care: The Development and Validation of the "Patient Reported Experiences and Outcomes of Safety in Primary Care" (PREOS-PC)

“…Skew is common, however, in questionnaires assessing patients' views of medical care 32,33 and does not necessarily limit the ability to reliably distinguish practices and patient subgroups with sufficient sample sizes such as ours. 34 The acceptability of the "Most recent safety problem" section was relatively low, with only 60% of eligible participants adequately completing that section. This could be partially explained by potentially unclear instructions in the branching question preceding that section.…”