Attempts to Limit Censoring in Measures of Patient Satisfaction

Abstract

Background: Measures of patient satisfaction are increasingly used to measure patient experience. Most satisfaction measures have notable ceiling effects, which limits our ability to learn from variation among relatively satisfied patients. This study tested a variety of single-question satisfaction measures for their mean overall score, ceiling and floor effect, and data distribution. In addition, we assessed the correlation between satisfaction and psychological factors and assessed how the various methods for measuring satisfaction affected net promoter scores (NPSs). Methodology: A total of 212 patients visiting orthopedic offices were enrolled in this randomized controlled trial. Patients were randomized to 1 of 5 newly designed, single-question satisfaction scales: (a) a helpfulness 11-point ordinal scale from 0 to 10, (b) a helpfulness ordinal 11-point scale from 0 to 5 (ie, with 1.5, 2.5, etc), (c) a helpfulness 100-point slider, (d) a satisfaction 11-point ordinal scale from 0 to 10, and (e) a willingness to recommend 11-point ordinal scale from 0 to 10. Additionally, patients completed the 2-item Pain Self-Efficacy Questionnaire (PSEQ-2), 5-item Short Health Anxiety Inventory (SHAI-5) Scale, and Patient-Reported Outcomes Measurement Information System (PROMIS) Depression. We assessed mean and median score, ceiling and floor effect, and skewness and kurtosis for each scale. Spearman’s correlation tests were used to test correlations between satisfaction and psychological status. Finally, we assessed the NPS for the various scales. Results: Ceiling effects ranged from 29% to 68%. The 11-point ordinal helpfulness scale from 0 to 10 had the least ceiling effect (29%). All of the scales were asymmetrically distributed, with the 11-point ordinal scale from 0 to 5 having the most Gaussian distribution (skew = 0.64 and kurtosis = 2.3). Satisfaction scores did not correlate with psychological factors: PSEQ-2 ( r = 0.04; P = .57), SHAI-5 ( r = 0.01; P = .93), and PROMIS Depression ( r = −0.04; P = .61). Net promoter scores varied substantially by scale design, with higher scores corresponding with greater ceiling effects. Conclusions: Variations in scale types, text anchors, and lead-in statements do not eliminate the ceiling effect of single-question measures of satisfaction with a visit to an orthopedic specialist. Further studies might test other scale designs and labels. Level of Evidence: Diagnostic; Level II

Does Adjusting for Social Desirability Reduce Ceiling Effects and Increase Variation of Patient-Reported Experience Measures?

“…The observation that there is no difference in satisfaction scores between patients who completed the social desirability instrument before or after the satisfaction measures suggests that its measurement has little influence on measurements of experience (PREMs). Three hundred one undergraduate students enrolled in a psychology course identified associations of increased reports of alcohol use with lower social desirability bias and greater religiosity, which was made greater (primed) by consideration of religiosity prior to reporting alcohol use ( 14 ). Our findings demonstrate that such priming effects do not occur when patients are first asked questions that measure their tendency to underreport negative attitudes (social desirability), perhaps because the questions in the scale were relatively abstract and did not have an apparent connection with ratings of satisfaction.…”

“…One study of 591 cocaine and opioid users in Baltimore documented an association between greater social desirability and (1) a lower likelihood of reporting recent drug use (measured on a 10-point ordinal scale) and (2) greater perceived social stigma (measured on a 17-point scale), accounting for symptoms of depression (4). Another study of 1,644 adolescents in Northwestern Burkina Faso (14) randomized people to a conventional verbal response arm (high social desirability bias environment) and a nonverbal response arm (low social desirability bias environment) and found significantly higher levels of reported sexual assault and forced sex in the non-verbal response (14). These findings suggest that social desirability could be part of the explanation of ceiling effects and nonparametric distributions of the current PREMs, but there are a variety of other modifiable and non-modifiable patient factors that may explain why patients are likely to give us top scores (15)(16)(17)(18)(19)(20)(21)(22)(23).…”

Patient reported satisfaction levels with the use of telemedicine for general surgery—A systematic review of randomized control trials

“…Firstly, the reliance upon patient satisfaction as a subjective outcome measure has many limitations, as previously outlined in depth by Ring et al [31] . Furthermore, satisfaction as an outcome measure can demonstrate the ‘ceiling effect’ [32] , which occurs when high scores are reported for both the intervention and control arms, limiting the ability to detect significant differences in outcomes (as clearly demonstrated by Cremedes et al where the median global satisfaction score was rated 5/5 for both telemedicine and control groups respectively [23] ). Secondly, several of the studies included in this systematic review failed to use validated instruments and tools to measure patient satisfaction, with heterogeneity in the instruments across the included studies.…”

Do Patient Experiences Have Mediating Roles on Patient Loyalty?

“… 20 , 27 Patients’ loyalty is directly affected by the positive experiences obtained throughout the healthcare services. 2 , 28 , 29 Since patient experience is the sum of interactions affecting patient perceptions, 30 different variables need to be examined from this perspective. 8 , 19 , 20 Complex relations between patient loyalty and patient experiences were analyzed through mediation analysis in this cross-sectional study.…”

“… 6 , 10 From the patient's perspective, communication with health professionals and receiving enough information about their treatment plans appear to be important factors in the patient experience. 28 , 29 …”