Language Features for Automated Evaluation of Cognitive Behavior Psychotherapy Sessions

Flemotomos, Nikolaos; Martínez, Victor R.; Gibson, James; Atkins, David C.; Creed, Torrey A.; Narayanan, Shrikanth

doi:10.21437/interspeech.2018-1518

Cited by 23 publications

(25 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Xiao et al (2015) found that the Support Vector Machines methods performed almost as well as trained coders. Similar results were reported in other studies (e.g., Flemotomos et al, 2018;Pérez-Rosas et al, 2019;Pérez-Rosas et al, 2017). Most studies only examined one type of method's performance.…”

Section: Article In Presssupporting

confidence: 88%

“…Pérez-Rosas et al, 2019 showed that as they increased the amount of data in their training set they observed significant improvement in prediction accuracy. Aligned with this finding, frequently observed codes (i.e., categories) in a dataset Howes et al, 2013;Perez-Rosas et al, 2017;Perez-Rosas et al, 2019;Xiao et al, 2015;Park et al, 2019;Flemotomos et al, 2018. Random Forest 7 Carcone et al, 2019Imel et al, 2015;Mieskes and Stiegelmayr, 2018;Blanchard et al, 2016a;Blanchard et al, 2016b;Donnelly et al, 2017;Wang et al, 2014.…”

Section: Article In Pressmentioning

confidence: 78%

“…In this context, the machine learning methods were designed to automatically assign codes from the behavioural coding measures to overt interactions recorded in the dataset (e.g., words/utteran- (Atkins et al, 2014;Can et al, 2015;Can et al, 2012;Can et al, 2016;Cao et al, 2020;Carcone et al, 2019, Study 1;Chakravarthula et al, 2015;Chen et al, 2019;Gibson et al, 2017;Gibson et al, 2016;Gupta et al, 2014;Hasan et al, 2019;Hasan et al, 2018;Imel et al, 2015;Perez-Rosas et al, 2017;Perez-Rosas et al, 2019;Singla et al, 2018;Tanana et al, 2016;Xiao et al, 2012;Xiao et al, 2015;Xiao, Can, et al, 2016;Xiao, Huang, et al, 2016) Medical care, provider-patient clinical interactions (Carcone et al, 2019, Study 2;Park et al, 2019) Education (teachers) (Blanchard et al, 2016a;Blanchard et al, 2016b;Donelly et al, 2017;Donnely et al, 2016a;Donnelly et al, 2016b;Samei et al, 2014;Samei et al, 2015;Song et al, 2020;Suresh et al, 2019;Wang et al, 2014) Counselling, (counsellors), (Althoff et al, 2016;Flemotomos et al, 2018;Gallo et al, 2015;Gaut et al, 2017;Goldberg et al, 2020…”

Section: Synthesis Of Resultsmentioning

confidence: 99%

See 2 more Smart Citations

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Ahmadi¹,

Noetel²,

Schellekens³

et al. 2021

Preprint

View full text Add to dashboard Cite

Many psychological treatments have been shown to be cost-effective and efficacious, as long as they are implemented faithfully. Assessing fidelity and providing feedback is expensive and time-consuming. Machine learning has been used to assess treatment fidelity, but the reliability and generalisability is unclear. We collated and critiqued all implementations of machine learning to assess the verbal behaviour of all helping professionals, with particular emphasis on treatment fidelity for therapists. We conducted searches using nine electronic databases for automated approaches of coding verbal behaviour in therapy and similar contexts. We completed screening, extraction, and quality assessment in duplicate. Fifty-two studies met our inclusion criteria (65.3% in psychotherapy). Automated coding methods performed better than chance, and some methods showed near human-level performance; performance tended to be better with larger data sets, a smaller number of codes, conceptually simple codes, and when predicting session-level ratings than utterance-level ones. Few studies adhered to best-practice machine learning guidelines. Machine learning demonstrated promising results, particularly where there are large, annotated datasets and a modest number of concrete features to code. These methods are novel, cost-effective, scalable ways of assessing fidelity and providing therapists with individualised, prompt, and objective feedback.

show abstract

Section: Article In Presssupporting

confidence: 88%

Section: Article In Pressmentioning

confidence: 78%

Section: Synthesis Of Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Ahmadi¹,

Noetel²,

Schellekens³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We also adapted it using in-domain psychotherapy data. For role assignment, the two diarized speaker clusters were assigned to either therapist or client roles, following the method presented in [37]. Performance of this system was evaluated on additional psychotherapy sessions provided by the counseling center.…”

Section: Methodsmentioning

confidence: 99%

Identifying Therapist and Client Personae for Therapeutic Alliance Estimation

et al. 2019

Self Cite

View full text Add to dashboard Cite

Psychotherapy, from a narrative perspective, is the process in which a client relates an ongoing life-story to a therapist. In each session, a client will recount events from their life, some of which stand out as more significant than others. These significant stories can ultimately shape one's identity. In this work we study these narratives in the context of therapeutic alliance-a self-reported measure on the perception of a shared bond between client and therapist. We propose that alliance can be predicted from the interactions between certain types of clients with types of therapists. To validate this method, we obtained 1235 transcribed sessions with client-reported alliance to train an unsupervised approach to discover groups of therapists and clients based on common types of narrative characters, or personae. We measure the strength of the relation between personae and alliance in two experiments. Our results show that (1) alliance can be explained by the interactions between the discovered character types, and (2) models trained on therapist and client personae achieve significant performance gains compared to competitive supervised baselines. Finally, exploratory analysis reveals important character traits that lead to an improved perception of alliance.

show abstract

“…This body of work has focused primarily on identifying therapist skills in Motivational Interviewing, a highly structured psychotherapy approach used for resolving ambivalence related to the treatment of conditions such as substance or alcohol use, or to engaging with treatment in general (Miller and Rollnick, 2012). Independently, Flemotomos et al 2018 and Rojas Barahona et al 2018 applied machine learning approaches to code behaviors common in the context of Cognitive Behavior Therapy (CBT). Rojas Barahona et al developed neural network models for classification of various types of client 'thinking errors' identified as part of cognitive behavioral treatment, while Flemotomos et al built SVM models to classify the overall quality of a CBT treatment session, looking at the distribution of different types of therapist behaviors used within the session, both process and content-oriented (e.g.…”

Section: Related Workmentioning

confidence: 99%

Identifying therapist conversational actions across diverse psychotherapeutic approaches

Lee¹,

Hull²,

Levine³

et al. 2019

Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology

View full text Add to dashboard Cite

While conversation in therapy sessions can vary widely in both topic and style, an understanding of the underlying techniques used by therapists can provide valuable insights into how therapists best help clients of different types. Dialogue act classification aims to identify the conversational "action" each speaker takes at each utterance, such as sympathizing, problem-solving or assumption checking. We propose to apply dialogue act classification to therapy transcripts, using a therapy-specific labeling scheme, in order to gain a high-level understanding of the flow of conversation in therapy sessions. We present a novel annotation scheme that spans multiple psychotherapeutic approaches, apply it to a large and diverse corpus of psychotherapy transcripts, and present and discuss classification results obtained using both SVM and neural network-based models. The results indicate that identifying the structure and flow of therapeutic actions is an obtainable goal, opening up the opportunity in the future to provide therapeutic recommendations tailored to specific client situations.

show abstract

Language Features for Automated Evaluation of Cognitive Behavior Psychotherapy Sessions

Cited by 23 publications

References 18 publications

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Identifying Therapist and Client Personae for Therapeutic Alliance Estimation

Identifying therapist conversational actions across diverse psychotherapeutic approaches

Contact Info

Product

Resources

About