Testing for response shift in treatment evaluation of change in self‐reported psychopathology amongst secondary psychiatric care outpatients

Abstract

ObjectivesIf patients change their perspective due to treatment, this may alter the way they conceptualize, prioritize, or calibrate questionnaire items. These psychological changes, also called “response shifts,” may pose a threat to the measurement of therapeutic change in patients. Therefore, it is important to test the occurrence of response shift in patients across their treatment.MethodsThis study focused on self‐reported psychological distress/psychopathology in a naturalistic sample of 206 psychiatric outpatients. Longitudinal measurement invariance tests were computed across treatment in order to detect response shifts.ResultsCompared with before treatment, post‐treatment psychopathology scores showed an increase in model fit and factor loading, suggesting that symptoms became more coherently interrelated within their psychopathology domains. Reconceptualization (depression/mood) and reprioritization (somatic and cognitive problems) response shift types were found in several items. We found no recalibration response shift.ConclusionThis study provides further evidence that response shift can occur in adult psychiatric patients across their mental health treatment. Future research is needed to determine whether response shift implies an unwanted potential bias in treatment evaluation or a desired cognitive change intended by treatment.

Re-evaluating randomized clinical trials of psychological interventions: Impact of response shift on the interpretation of trial results

“…The detected response shift was not found to be associated with diabetes distress, and only marginally increased overall treatment effectiveness. Occurrence of response shift in measures of depression has been evidenced in previous studies too [10][11][12][13]41]. Similar to our findings, some studies indicated response shift in measures like depressive affect, where Carlier [13] detected reconceptualization of mood items, Smith et al [12] found reprioritization of the item 'how often do you feel depressed', and Wu [11] found considerable recalibration response shift in items of the negative attitude factor.…”

“…Finally, Carlier at al. [ 13 ] investigated response shift in self-reported psychological distress from psychiatric outpatients who received psychotherapy (mainly CBT), pharmacotherapy, or a combination of both. They found several indications of reprioritization and reconceptualization, but these response shifts did not impact the assessment of treatment effectiveness.…”

The Spanish Version of the Sexual Opinion Survey (SOS-6): Evidence of Validity of a Short Version

“…SOS-6 was established as a factor with six observable variables between errors. The Weighted Least Squares Means and Variance Adjusted (WLSMV; Beauducel & Herzberg, 2006 ; Carlier et al, 2019 ; Hirschfeld & von Brachel, 2014 ) estimation method was used; the WLSMV estimation method is a robust estimator in the absence of multivariate normality and is designed for ordinal data ( Hu & Bentler, 2016 ). The following criteria were taken into account to evaluate the multigroup CFA: CFI and TLI> .90 and RMSEA <.08 ( Hu & Bentler, 1999 ; Manrique & Semenova, 2015 ; Wang & Wang, 2012 ), as well as the difference between the CFI comparative index values ( Bentler, 1990 ).…”

Evaluating Response Shift in Statistical Mediation Analysis

“…Severe; not measuring same construct Carlier et al (2019) found in a sample of individuals receiving outpatient psychiatric treatment (multiple diagnoses) that items assessing suicidal ideation and hopelessness broke apart from the mood subscale to form a distinct factor, which was different from the structure of the items at pretreatment. The authors concluded that these concepts became more distinct after treatment, whereas the other mood-related items did not.…”

“…Furthermore, Sprangers and Schwartz (1999) defined catalysts as changes in health status (e.g., an intervention, elapsed time, a diagnosis, or medical procedure) and mechanisms as behavioral, cognitive, and affective processes that accommodate the catalyst (e.g., coping or social comparison) but are unrelated to true change in construct. Their theoretical model of response shift proposes that a catalyst triggers may not be measuring same construct Carlier et al (2019) found response shift in a sample of individuals receiving outpatient treatment in two items-one cognitive ("I could not concentrate well") and one somatic ("I was shaking or trembling") such that the factor loadings were higher after treatment than before. The authors concluded that the patients placed more value on these problems at posttreatment.…”