Overcoming barriers to the adoption and implementation of predictive modeling and machine learning in clinical care: what can we learn from US academic medical centers?

Abstract

There is little known about how academic medical centers (AMCs) in the US develop, implement, and maintain predictive modeling and machine learning (PM and ML) models. We conducted semi-structured interviews with leaders from AMCs to assess their use of PM and ML in clinical care, understand associated challenges, and determine recommended best practices. Each transcribed interview was iteratively coded and reconciled by a minimum of 2 investigators to identify key barriers to and facilitators of PM and ML adoption and implementation in clinical care. Interviews were conducted with 33 individuals from 19 AMCs nationally. AMCs varied greatly in the use of PM and ML within clinical care, from some just beginning to explore their utility to others with multiple models integrated into clinical care. Informants identified 5 key barriers to the adoption and implementation of PM and ML in clinical care: (1) culture and personnel, (2) clinical utility of the PM and ML tool, (3) financing, (4) technology, and (5) data. Recommendation to the informatics community to overcome these barriers included: (1) development of robust evaluation methodologies, (2) partnership with vendors, and (3) development and dissemination of best practices. For institutions developing clinical PM and ML applications, they are advised to: (1) develop appropriate governance, (2) strengthen data access, integrity, and provenance, and (3) adhere to the 5 rights of clinical decision support. This article highlights key challenges of implementing PM and ML in clinical care at AMCs and suggests best practices for development, implementation, and maintenance at these institutions.

Technology Acceptance of a Machine Learning Algorithm Predicting Delirium in a Clinical Setting: a Mixed-Methods Study

“…Several barriers and concerns have been raised for the implementation of ML-based predictive models in clinical decision support systems [5][6][7][8]. As the final decision is always the responsibility of the user, it is crucial to open the often criticized black box of ML decisions so that healthcare professionals can detect bias or error [9].…”

The potential of artificial intelligence to improve patient safety: a scoping review

“…Once high-quality AI solutions are developed, additional factors beyond performance must be considered to increase the likelihood of successful implementation and adoption by individual providers. There is an active area of research focused on identifying key barriers and facilitators to implementation of AI-based tools in healthcare 78 , 80 , 81 .…”

Patient Similarity Analytics for Explainable Clinical Risk Prediction

“…established guidelines, attention is shifting toward improving their explainability and interpretability (9-19). Explainability is de ned as the extent of which a model's prediction process can be described, while interpretability is de ned as the degree to which a user can understand the predictions made by a model (20)(21)(22).…”

“…established guidelines, attention is shifting toward improving their explainability and interpretability (9-19). Explainability is de ned as the extent of which a model's prediction process can be described, while interpretability is de ned as the degree to which a user can understand the predictions made by a model (20)(21)(22).Recently, patient similarity analytics has become a popular technique for CRPM (23). The underlying concept is to identify similar patients to a patient of interest, and use them as a clinically meaningful subgroup to derive more precise prognostic information (24), and has also been shown to improve prediction accuracy (28,29).…”