Using a simulation centre to evaluate preliminary acceptability and impact of an artificial intelligence-powered clinical decision support system for depression treatment on the physician–patient interaction

Abstract

Background Recently, artificial intelligence-powered devices have been put forward as potentially powerful tools for the improvement of mental healthcare. An important question is how these devices impact the physician-patient interaction. Aims Aifred is an artificial intelligence-powered clinical decision support system (CDSS) for the treatment of major depression. Here, we explore the use of a simulation centre environment in evaluating the usability of Aifred, particularly its impact on the physician–patient interaction. Method Twenty psychiatry and family medicine attending staff and residents were recruited to complete a 2.5-h study at a clinical interaction simulation centre with standardised patients. Each physician had the option of using the CDSS to inform their treatment choice in three 10-min clinical scenarios with standardised patients portraying mild, moderate and severe episodes of major depression. Feasibility and acceptability data were collected through self-report questionnaires, scenario observations, interviews and standardised patient feedback. Results All 20 participants completed the study. Initial results indicate that the tool was acceptable to clinicians and feasible for use during clinical encounters. Clinicians indicated a willingness to use the tool in real clinical practice, a significant degree of trust in the system's predictions to assist with treatment selection, and reported that the tool helped increase patient understanding of and trust in treatment. The simulation environment allowed for the evaluation of the tool's impact on the physician–patient interaction. Conclusions The simulation centre allowed for direct observations of clinician use and impact of the tool on the clinician–patient interaction before clinical studies. It may therefore offer a useful and important environment in the early testing of new technological tools. The present results will inform further tool development and clinician training materials.

Evaluating the Clinical Feasibility of an Artificial Intelligence–Powered, Web-Based Clinical Decision Support System for the Treatment of Depression in Adults: Longitudinal Feasibility Study

“…The results of the patient baseline questionnaire scores are summarized in Table 1. Patients identified other clinical comorbidities, such as migraines (2/14, 14% of patients) and anxiety disorders (7/14, 50% of patients), which are described in detail in Multimedia Appendix 1 [10,11,20,[25][26][27][28][29][30][31][32], Table S2.…”

“…Following a previous simulation center study [10] and ahead of larger clinical trials aimed at assessing safety and effectiveness, we decided to conduct a feasibility study aimed at exploring the feasibility of the CDSS in a real clinical setting and to assess its longitudinal impact on the patient-clinician relationship. This study has 4 aims: 1. To assess the feasibility of the CDSS for use in clinical practice 2.…”

“…For example, researchers found that the dropout rate was significantly lower—as low as 12%—in apps offering human feedback and in-app mood monitoring [ 6 ]. In addition, previous decision support systems have demonstrated the need for a tool that provides real-time utility [ 7 ] and the ability to personalize treatment choices and differentiate between medications [ 8 ] in a quantifiable manner [ 9 ] and incorporates clinical practice guidelines [ 10 ].…”

“…However, there are no easily accessible point-of-care tools available to aid in the optimization of treatment success and minimize the time to remission. Furthermore, treatments are essentially equally effective at the population level; as such, to improve outcomes, treatment selection must address the individual’s specific characteristics [ 10 , 12 ]. As such, there is a clear need for improved and personalized decision support for mental health care [ 13 ].…”

“…The tool incorporates a deep-learning model that was validated and trained on clinical and demographic baseline data to support treatment selection by providing individualized probabilities of remission for specific treatment options. Please see the study by Benrimoh et al [ 10 ] for a description of the tool, and the studies by Mehltretter et al [ 4 , 14 ] for a description of the machine learning model and model training and validation methodology. Clinicians accessing the app first see their patient’s self-report questionnaire scores and can examine these through graphs showing trends over time or at the individual item level.…”