Can machine learning complement traditional medical device surveillance? A case-study of dual-chamber implantable cardioverter–defibrillators

Abstract

BackgroundMachine learning methods may complement traditional analytic methods for medical device surveillance.Methods and resultsUsing data from the National Cardiovascular Data Registry for implantable cardioverter–defibrillators (ICDs) linked to Medicare administrative claims for longitudinal follow-up, we applied three statistical approaches to safety-signal detection for commonly used dual-chamber ICDs that used two propensity score (PS) models: one specified by subject-matter experts (PS-SME), and the other one by machine learning-based selection (PS-ML). The first approach used PS-SME and cumulative incidence (time-to-event), the second approach used PS-SME and cumulative risk (Data Extraction and Longitudinal Trend Analysis [DELTA]), and the third approach used PS-ML and cumulative risk (embedded feature selection). Safety-signal surveillance was conducted for eleven dual-chamber ICD models implanted at least 2,000 times over 3 years. Between 2006 and 2010, there were 71,948 Medicare fee-for-service beneficiaries who received dual-chamber ICDs. Cumulative device-specific unadjusted 3-year event rates varied for three surveyed safety signals: death from any cause, 12.8%–20.9%; nonfatal ICD-related adverse events, 19.3%–26.3%; and death from any cause or nonfatal ICD-related adverse event, 27.1%–37.6%. Agreement among safety signals detected/not detected between the time-to-event and DELTA approaches was 90.9% (360 of 396, k=0.068), between the time-to-event and embedded feature-selection approaches was 91.7% (363 of 396, k=−0.028), and between the DELTA and embedded feature selection approaches was 88.1% (349 of 396, k=−0.042).ConclusionThree statistical approaches, including one machine learning method, identified important safety signals, but without exact agreement. Ensemble methods may be needed to detect all safety signals for further evaluation during medical device surveillance.

Medical device active surveillance of spontaneous reports: A literature review of signal detection methods

“…Among the five published methods articles in device registries, CUSUM and survival analysis are the earliest described examples while propensity scoring using the Data Extraction and Longitudinal Trend Analysis (DELTA) framework is described in more recent years . The DELTA framework may be considered the closest analog to a desired future state for continuous surveillance of device spontaneous reports.…”

“…PV methods groups include Disproportionality , Propensity Score , Risk Adjustment , Bayesian , Sequential Probability Ratio , and Advanced Data Mining . Among the 14 reviewed Device articles, methods groups ordered by number of articles were Disproportionality , Risk Adjustment , Sequential Probability Ratio , Survival Modeling , Propensity Score , Bayesian, and Advanced Data Mining . Changepoint analysis from one Device article by Xu and other methods that did not fall into any method group were labeled as Other .…”

Robotics in cardiac surgery

“…48 Machine learning tools can be applied to complement traditional analytical methods to provide real-time, evidence-based, personalised answers. 49 In this way, the gap between implementation and evidence can be bridged without delaying the process or compromising patient safety. An international registry infrastructure for cardiovascular When it comes to novel surgical technologies, perception can often act as a more powerful driver than evidence in promoting adoption device post-market surveillance was established in 2011 by the FDA, 47 although there is not yet a dedicated international registry for robotics in cardiac surgery.…”

Quantifying the utilization of medical devices necessary to detect postmarket safety differences: A case study of implantable cardioverter defibrillators

“…To ensure the accuracy of assigned ICD type, manufacturer, and model, for each ICD generator implanted during the study period, the device type (single-chamber, dual-chamber, and cardiac resynchronization therapy-defibrillator) and listed manufacturer and model name and number within the NCDR-ICD registry were reviewed and verified, as described in prior work. 16 Once accuracy of the ICD type, manufacturer, and model were confirmed, a study ID for the model was generated and used to ensure that the study team was blinded to the device manufacturer and model in accordance with our Data Use Agreement with the American College of Cardiology. Given our interest in medical device utilization needed to detect safety differences among device models, only models with at least 20 implantations were included in our study.…”