Observational Coding Training Methods for CBT Treatment Fidelity: A Systematic Review

Rodríguez‐Quintana, Natalie; Lewis, Cara C.

doi:10.1007/s10608-018-9898-5

Cited by 19 publications

(21 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Automated coding methods showed promising results in analysing helping professionals' language, so they can be used to provide feedback and improve practitioners' skills. Our systematic review shows that automated coding methods provided accurate estimation of treatment fidelity, including all three components (adherence, differentiation, and competence; Rodriguez-Quintana & Lewis, 2018). In motivational interviewing, for example, automated methods were able to code adherence to therapeutic strategies (e.g., affirming change), differentiation of proscribed strategies (e.g., use of closed questions; Tanana et al, 2016), and competence in delivery (e.g., session-level empathy ratings; Gibson et al, 2016).…”

Section: Potential Applicationsmentioning

confidence: 96%

“…Fidelity encompasses three core components: adherence, differentiation, and competence (Rodriguez-Quintana & Lewis, 2018). Adherence describes a therapist's use of methods proposed by the guiding framework (e.g., using cognitive defusion while delivering Acceptance and Commitment Therapy).…”

Section: Article In Pressmentioning

confidence: 99%

“…Their clients are more likely to complete treatment and change behaviour too (Golin et al, 2002;Street et al, 2009). As a result, researchers have developed a range of treatment integrity measures (Rodriguez-Quintana & Lewis, 2018), including many that assess the content of therapy (McGlinchey & Dobson, 2003) and the process of therapy (e.g., Motivational Interviewing Skill Code: Miller et al, 2003;Motivational Interviewing Treatment Integrity: Moyers, Martin, et al, 2005). There are even measures for assessing how well treatment fidelity is assessed (Perepletchikova et al, 2009).…”

Section: Article In Pressmentioning

confidence: 99%

See 2 more Smart Citations

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Ahmadi¹,

Noetel²,

Schellekens³

et al. 2021

Preprint

View full text Add to dashboard Cite

Many psychological treatments have been shown to be cost-effective and efficacious, as long as they are implemented faithfully. Assessing fidelity and providing feedback is expensive and time-consuming. Machine learning has been used to assess treatment fidelity, but the reliability and generalisability is unclear. We collated and critiqued all implementations of machine learning to assess the verbal behaviour of all helping professionals, with particular emphasis on treatment fidelity for therapists. We conducted searches using nine electronic databases for automated approaches of coding verbal behaviour in therapy and similar contexts. We completed screening, extraction, and quality assessment in duplicate. Fifty-two studies met our inclusion criteria (65.3% in psychotherapy). Automated coding methods performed better than chance, and some methods showed near human-level performance; performance tended to be better with larger data sets, a smaller number of codes, conceptually simple codes, and when predicting session-level ratings than utterance-level ones. Few studies adhered to best-practice machine learning guidelines. Machine learning demonstrated promising results, particularly where there are large, annotated datasets and a modest number of concrete features to code. These methods are novel, cost-effective, scalable ways of assessing fidelity and providing therapists with individualised, prompt, and objective feedback.

show abstract

Section: Potential Applicationsmentioning

confidence: 96%

Section: Article In Pressmentioning

confidence: 99%

Section: Article In Pressmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Ahmadi¹,

Noetel²,

Schellekens³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A integridade ou fidelidade do tratamento se refere ao quanto o indivíduo conduziu e seguiu a abordagem ou metodologia proposta de forma habilidosa. Avaliar os terapeutas, então, é um dado importante para uma estimativa mais fidedigna do tamanho do efeito da intervenção e esse processo se dá pela avaliação da integridade do tratamento (Rodriguez-Quintana & Lewis, 2018). Quando esse aspecto não é avaliado no estudo, a validade interna do mesmo pode ficar comprometida (McGrath & Forman, 2012).…”

Section: Introductionunclassified

“…Quando esse aspecto não é avaliado no estudo, a validade interna do mesmo pode ficar comprometida (McGrath & Forman, 2012). A integridade do tratamento pode ser avaliada por meio da aderência e da competência dos terapeutas (Rodriguez-Quintana & Lewis, 2018). A aderência se refere ao quanto o terapeuta seguiu a intervenção ou abordagem teórica proposta e a competência diz respeito ao quanto o terapeuta foi habilidoso em seu desempenho (Plumb & Vilardaga, 2010).…”

Section: Introductionunclassified

Aderência e Competência de Terapeutas a um Programa para Redução de Estresse Baseado na Terapia de Aceitação e Compromisso

Dillenburg

Ludwig²,

Garicochêa

et al. 2021

ctc

View full text Add to dashboard Cite

A Integridade do Tratamento (IT) é um conceito que se refere à avaliação da competência e aderência a um tratamento ou intervenção. Aderência e competência podem ser definidas respectivamente como o quanto o terapeuta seguiu o protocolo ou abordagem teórica proposta e o quanto ele foi habilidoso no seu desempenho. Apesar da importância deste dado, a avaliação da IT ainda é frequentemente omitida nas publicações científicas. O objetivo deste artigo foi avaliar aderência e competência de terapeutas em uma intervenção de oito encontros para redução de estresse em universitários baseada na Terapia de Aceitação e Compromisso. Os encontros da intervenção foram gravados e os encontros selecionados para o estudo foram avaliados por três juízes experts em ACT. Para análise dos dados, foi utilizado o coeficiente Fleiss kappa e o teste de Friedman. Os resultados indicaram boa concordância entre os juízes. Os terapeutas mostraram uma ótima aderência ao programa (mínimo 77,8%) e bons níveis de competência (2,7 a 3 numa escala de 0 a 3). Em relação aos processos de flexibilidade psicológica, os níveis de aderência foram maiores para os processos enfocados em cada encontro. Sugere-se estudos futuros com amostras maiores que possibilitem diferentes análises para entender o fenômeno da IT.

show abstract

Treatment fidelity in eating disorders and psychological research: Current status and future directions

Peyser

Sysko

Webb

et al. 2021

Intl J Eating Disorders

View full text Add to dashboard Cite

Objective: The evaluation and use of treatment fidelity procedures are rare in the development and implementation of psychological interventions. This article aims to review the construct of treatment fidelity, highlight limitations to currently available measures, and introduce a conceptual framework for studying and adapting fidelity measures in clinical research and practice using eating disorders as an example.Method: As treatment fidelity assesses whether an intervention was delivered as intended, we operationalized this construct as: (a) treatment adherence, (b) therapist competence, and (c) treatment differentiation.Results: There is a significant gap in the literature assessing and documenting treatment fidelity. Available studies indicate that existing adherence measures can be time consuming, costly, and are not widely used in the field. Furthermore, therapist competence is a complex and context-dependent construct that is challenging to measure. Finally, treatment differentiation is often inferred by ensuring adherence.Discussion: The development of simplified formal tests of treatment fidelity would help draw conclusions about treatment efficacy and improve the dissemination and implementation of interventions to promote optimal clinical outcomes.

show abstract

Observational Coding Training Methods for CBT Treatment Fidelity: A Systematic Review

Cited by 19 publications

References 35 publications

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Aderência e Competência de Terapeutas a um Programa para Redução de Estresse Baseado na Terapia de Aceitação e Compromisso

Treatment fidelity in eating disorders and psychological research: Current status and future directions

Contact Info

Product

Resources

About