Psychometric Properties of the Norwegian Version of the Cognitive Therapy Adherence and Competence Scale (CTACS) and Its Associations With Outcomes Following Treatment in IAPT Norway

Abstract

Background: No studies have examined the underlying structure or predictive validity of the Cognitive Therapy Adherence and Competence Scale (CTACS). Examining the structure of the CTACS is of great relevance because it could provide information on what constitutes competence in CBT, and whether some underlying factors are more important for predicting treatment outcomes than others. This study investigates the psychometric properties of the Norwegian version of CTACS and its associations with treatment outcomes in a sample of primary care clients who received CBT for anxiety and/or depression.Method: Independent assessors rated audiotaped therapy sessions (early, mid and late in treatment) in a sample of 132 primary care clients (mean [SD] age = 34.8 [11.8], 63.6% women), participating in the Prompt Mental Health Care trial. Outcomes were symptoms of anxiety and depression assessed by patient self-report questionnaires. Structural validity was examined by means of confirmatory and exploratory factor analyses (CFA/EFA), whereas longitudinal associations with treatment outcome were explored by adopting multilevel modeling.Results: No evidence was found for the divergent validity of the constructs competence and adherence as indicated by a very high correlation between these two subscales in CTACS (0.97). Regarding reliability, ICCs for the mean score of the full competence scale and its associated subscales were generally good to excellent (0.70–0.80), although the subscale measuring the quality of the therapeutic relationship was relatively low (0.44). Internal consistency was overall acceptable, but our CFA models did not provide an acceptable fit for the pre-specified one-factor and four-factor solutions. EFA results were difficult to interpret, with a sub-optimal three-factor solution providing best model fit and only two meaningful factors [CBT specific skills (α = 0.82) and session structure (α = 0.59)]. Overall, the results indicated no evidence for the scales' predictive validity.Conclusion: Our findings point to several psychometric problems of the CTACS that may limit both its research and clinical utility. The importance of providing empirical evidence for both reliability and validity aspects of scales are discussed and suggestions for future research are provided.

Evaluating Delivery of a CBT-Based Group Intervention for Schoolchildren With Emotional Problems: Examining the Reliability and Applicability of a Video-Based Adherence and Competence Measure

“…This implies that either it was difficult to come to an agreement regarding these items, or there was something with the instrument that made it difficult to calibrate and reach consensus when scoring these items. As Lervik et al ( 2021 ) also suggests, it is probably more difficult to score and interpret interpersonal relationships and the more abstract items, as opposed to more structural and concrete parts of a CBT-intervention (e.g., checking homework assignments, or putting up an agenda). Although the scoring team discussed the content and meaning of each item all along, a thorough operationalization beforehand could have provided even more accurate assessments.…”

A study of self-perceived change in CBT competence among Norwegian therapists undertaking a 2-year CBT training for working with young people

“…While the scale can measure both adherence and competence, this study only collected competence scores. According to the psychometric evaluation of the Norwegian version of the CTACS by Lervik et al (2021) there is no evidence for divergent validity between the adherence and competence scale. The study reported acceptable internal consistency for the full scale (α = 0.91) and acceptable to low internal consistency for the subscales structure (α = 0.76), therapeutic relationship (α = 0.84), and CBT techniques (α = 0.73) and conceptualization (α = 0.61).…”