Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study

Abstract

Background Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature. Objective This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care. Methods In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch. Results Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch. Conclusions Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.

Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study

“…DUH is the flagship hospital of a multi-hospital academic health system with approximately 80,000 emergency department (ED) visits annually. According to our institutional definition for sepsis, over 20% of adults admitted through the DUH ED develop sepsis [ 31 ], and nearly 68% of sepsis occurs within the first 24 hours of hospital encounter [ 32 ].…”

“…Both nurses and physicians were educated on the model’s aggregate performance measures relative to other methods, and visualizations of individual patient cases were presented to demonstrate how the model could detect sepsis hours before the clinical diagnosis [ 30 ]. A full description of the planning and implementation process can be found elsewhere [ 32 ].…”

Challenges of developing artificial intelligence‐assisted tools for clinical medicine

“…There is a growing recognition of the critical role of implementation into the clinical process for AI tools, with the goal to “design the best possible care delivery system for a given problem.” 4–6 This is usually an iterative process that goes through the delivery process before, during, and after implementation with the AI tool and focuses on designing and improving user interfaces.…”

“…In gastroenterology, practitioners are specialists who are experts in the field and should have the ability to verify the system performance. Use of the AI tools must consider the balance of power, in particular how the AI tools may impinge on professional authority for clinicians 5 . Furthermore, by understanding how the prediction is made, practitioners would be able to assess if the prediction is being generated from actual signal or is being distorted by confounding variables.…”

Artificial Intelligence–Aided Precision Medicine for COVID-19: Strategic Areas of Research and Development (Preprint)

“…For example, Duke University adopted a system called Sepsis Watch that identifies in advance the inflammation leading to sepsis-one of the leading causes of hospital deaths. Within two years from the tool introduction, the number of sepsis-induced patients drastically decreased [25], thanks to three key elements: (1) adaptation of the predictive model to a highly specific context; (2) scalability through integration with hospital workflows; and (3) the adopted user experience-based approach, which places clinicians and health care professionals at the center of the software development process, adhering with the human-in-the-loop paradigm [26,27].…”