Evaluating the perceived utility of an artificial intelligence-powered clinical decision support system for depression treatment using a simulation center

Tanguay-Sela, Myriam; Benrimoh, David; Popescu, Christina; Pérez, Tamara; Rollins, Colleen; Snook, Emily; Lundrigan, Eryn; Armstrong, Caitrin; Perlman, Kelly; Fratila, Robert; Mehltretter, Joseph; Israel, Sonia; Champagne, Monique; Williams, Jeremy S.; Simard, Jade; Parikh, Sagar V.; Karp, Jordan F.; Heller, Katherine; Linnaranta, Outi; Cardona, Liliana Gómez; Turecki, Gustavo; Margolese, Howard C.

doi:10.1016/j.psychres.2021.114336

Cited by 32 publications

(53 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aifred Health [2][3][4][5] has designed a machine-learning based clinical decision support system (CDSS) for physicians treating patients with depression. Patient characteristics of interests such as sociodemographic information, clinical information and medical history are input into the CDSS.…”

Section: Motivating Settingmentioning

confidence: 99%

“…However, some interventions naturally operate at a group level, or target either a social network or physical environment. For example, in a trial assessing the clinical utility, safety, and potential effectiveness of a machine-learning based clinical decision support system (CDSS) developed by Aifred Health [2][3][4][5], the decision support tool is designed for physicians, naturally forming clusters of individual patients being treated by that physician [6,7]. For these types of interventions, a population-level effect is of more interests to researchers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

New designs for Bayesian adaptive cluster-randomized trials

Shen¹,

Golchi²,

Moodie³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Adaptive approaches, allowing for more flexible trial design, have been proposed for individually randomized trials to save time or reduce sample size. However, adaptive designs for cluster-randomized trials in which groups of participants rather than individuals are randomized to treatment arms are less common. Motivated by a cluster-randomized trial designed to assess the effectiveness of a machine-learning based clinical decision support system for physicians treating patients with depression, two Bayesian adaptive designs for cluster-randomized trials are proposed to allow for early stopping for efficacy at pre-planned interim analyses. The difference between the two designs lies in the way that participants are sequentially recruited. Given a maximum number of clusters as well as maximum cluster size allowed in the trial, one design sequentially recruits clusters with the given maximum cluster size, while the other recruits all clusters at the beginning of the trial but sequentially enrolls individual participants until the trial is stopped early for efficacy or the final analysis has been reached. The design operating characteristics are explored via simulations for a variety of scenarios and two outcome types for the two designs. The simulation results show that for different outcomes the design choice may be different. We make recommendations for designs of Bayesian adaptive cluster-randomized trial based on the simulation results.

show abstract

Section: Motivating Settingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

New designs for Bayesian adaptive cluster-randomized trials

Shen¹,

Golchi²,

Moodie³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…One of the many fields to which AI-CDSS are currently being applied is the treatment of major depressive disorder (MDD) (Benrimoh et al, 2018; Kleinerman et al, 2021; Mehltretter et al, 2020, 2019; Tanguay-Sela et al, 2022) Depression is highly prevalent in the general population and is associated with significant morbidity and mortality (Lépine and Briley, 2011). Despite its high prevalence and the availability of many effective treatment modalities, only a third of patients will reach remission after their first treatment (Warden et al, 2007).…”

Section: Introductionmentioning

confidence: 99%

“…Despite its high prevalence and the availability of many effective treatment modalities, only a third of patients will reach remission after their first treatment (Warden et al, 2007). Antidepressants are frequently prescribed using an educated trial-and-error approach (Tomlinson et al, 2020) in which patients typically undergo several rounds of different antidepressants before achieving remission, and there are no current widely-used tools which can improve the match between patients and antidepressants at the point of care without expensive additional testing (Lépine and Briley, 2011; Tanguay-Sela et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

“…In an effort to improve decision making, researchers have developed artificial intelligence models that predict treatment-specific probabilities of remission for individual patients (Kleinerman et al, 2021; Mehltretter et al, 2020, 2019). These tools were integrated into CDSS and found to have the potential to improve treatment outcomes (Benrimoh et al, 2021; Lépine and Briley, 2011; Tanguay-Sela et al, 2022). In addition, recent work has shown potential in predicting the occurrence of side effects in response to various treatments based on genetic, clinical, and demographic data (Busch and Menke, 2019; Taliaz et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Mixed Methods Comparison of Artificial Intelligence-Powered Clinical Decision Support System Interfaces for Multiple Criteria Decision Making in Antidepressant Selection

Kleinerman‬‏

Benrimoh

Golden

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

BACKGROUND: Artificial intelligence-powered clinical decision support systems (AI-CDSS) have recently become foci of research. When clinicians face decisions about treatment selection, they must contemplate multiple criteria simultaneously. The relative importance of these criteria often depends on the clinical scenario, as well as clinician and patient preferences. It remains unclear how AI-CDSS can optimally assist clinicians in making these complex decisions. In this work we explore clinician reactions to different presentations of AI results in the context of multiple criteria decision-making during treatment selection for major depressive disorder. METHODS: We developed an online platform for depression treatment selection to test three interfaces. In the probabilities alone (PA) interface, we presented probabilities of remission and three common side effects for five antidepressants. In the clinician-determined weights (CDW) interface, participants assigned weights to each of the outcomes and obtained a score for each treatment. In the expert-derived weights interface (EDW), outcomes were weighted based on expert opinion. Each participant completed three clinical scenarios, and each scenario was randomly paired with one interface. We collected participants' impressions of the interfaces via questionnaires and written and verbal feedback. RESULTS: Twenty-two physicians completed the study. Participants felt that the CDW interface was most clinically useful (H=10.29, p<0.01) and more frequently reported that it had an impact on their decision making (PA: in 55.5% of experienced scenarios, CDW: in 59.1%, EDW: in 36.6%). Clinicians most often chose a treatment different from their original choice after reading the clinical scenario in the CDW interface (PA: 26.3%, CDW: 33.3%, EDW: 15.8%). CONCLUSION: Clinicians found a decision support interface where they could set the weights for different potential outcomes most useful for multi-criteria decision making. Allowing clinicians to weigh outcomes based on their expertise and the clinical scenario may be a key feature of a future clinically useful multi-criteria AI-CDSS.

show abstract

Model‐Informed Precision Dosing Using Machine Learning for Levothyroxine in General Practice: Development, Validation and Clinical Simulation Trial

Janssen Daalen,

Doesburg,

Hunik

et al. 2024

Clin Pharma and Therapeutics

View full text Add to dashboard Cite

Levothyroxine is one of the most prescribed drugs in the western world. Dosing is challenging due to high‐interindividual differences in effective dosage and the narrow therapeutic window. Model‐informed precision dosing (MIPD) using machine learning could assist general practitioners (GPs), but no such models exist for primary care. Furthermore, introduction of decision‐support algorithms in healthcare is limited due to the substantial gap between developers and clinicians' perspectives. We report the development, validation, and a clinical simulation trial of the first MIPD application for primary care. Stable maintenance dosage of levothyroxine was the model target. The multiclass model generates predictions for individual patients, for different dosing classes. Random forest was trained and tested on a national primary care database (n = 19,004) with a final weighted AUC across dosing options of 0.71, even in subclinical hypothyroidism. TSH, fT4, weight, and age were most predictive. To assess the safety, feasibility, and clinical impact of MIPD for levothyroxine, we performed clinical simulation studies in GPs and compared MIPD to traditional prescription. Fifty‐one GPs selected starting dosages for 20 primary hypothyroidism cases without and then with MIPD 2 weeks later. Overdosage and underdosage were defined as higher and lower than 12.5 μg relative to stable maintenance dosage. MIPD decreased overdosage in number (30.5 to 23.9%, P < 0.01) and magnitude (median 50 to 37.5 μg, P < 0.01) and increased optimal starting dosages (18.3 to 30.2%, P < 0.01). GPs considered lab results more often with MIPD and most would use the model frequently. This study demonstrates the clinical relevance, safety, and effectiveness of MIPD for levothyroxine in primary care.

show abstract

Evaluating the perceived utility of an artificial intelligence-powered clinical decision support system for depression treatment using a simulation center

Cited by 32 publications

References 31 publications

New designs for Bayesian adaptive cluster-randomized trials

New designs for Bayesian adaptive cluster-randomized trials

A Mixed Methods Comparison of Artificial Intelligence-Powered Clinical Decision Support System Interfaces for Multiple Criteria Decision Making in Antidepressant Selection

Model‐Informed Precision Dosing Using Machine Learning for Levothyroxine in General Practice: Development, Validation and Clinical Simulation Trial

Contact Info

Product

Resources

About