Initial Validation of a Machine Learning-Derived Prognostic Test (KidneyIntelX) Integrating Biomarkers and Electronic Health Record Data To Predict Longitudinal Kidney Outcomes

Chauhan, Kinsuk; Nadkarni, Girish N.; Fleming, Fergus; McCullough, James R.; He, Cijiang; Quackenbush, John; Murphy, Barbara; Donovan, Michael; Coca, Steven G.; Bonventre, Joseph V.

doi:10.34067/kid.0002252020

Cited by 20 publications

(18 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These biomarkers have demonstrated reliable independent prognostic signals for kidney function [11,12,15,[35][36][37][38]. In our previous study, we found that including biomarkers to clinical data derived from EHR at a single-centre had better predictive performance than clinical models alone [13]. However, that study included few patients with prevalent CKD (approximately one third had CKD in the cohort with type 2 diabetes and one quarter had CKD in the APOL1 high-risk cohort).…”

Section: Discussionmentioning

confidence: 96%

“…Machine learning can combine biomarkers and EHR data to produce prognostic risk scores. We previously demonstrated that combining biomarkers and EHR data in patients with type 2 diabetes and APOL-1 highrisk genotype improved prediction of kidney outcomes over clinical models [13]. A simple risk score that improves the ability to identify patients with DKD at low, intermediate, and high risk of progressive decline in kidney function has the potential to improve outcomes through more effective use of medications and efficient resource allocation at the primary care physician level.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Derivation and validation of a machine learning risk score using biomarker and electronic patient data to predict progression of diabetic kidney disease

Chan

Nadkarni

Fleming³

et al. 2021

Diabetologia

Self Cite

111

View full text Add to dashboard Cite

Aim Predicting progression in diabetic kidney disease (DKD) is critical to improving outcomes. We sought to develop/validate a machine-learned, prognostic risk score (KidneyIntelX™) combining electronic health records (EHR) and biomarkers. Methods This is an observational cohort study of patients with prevalent DKD/banked plasma from two EHR-linked biobanks. A random forest model was trained, and performance (AUC, positive and negative predictive values [PPV/NPV], and net reclassification index [NRI]) was compared with that of a clinical model and Kidney Disease: Improving Global Outcomes (KDIGO) categories for predicting a composite outcome of eGFR decline of ≥5 ml/min per year, ≥40% sustained decline, or kidney failure within 5 years. Results In 1146 patients, the median age was 63 years, 51% were female, the baseline eGFR was 54 ml min−1 [1.73 m]−2, the urine albumin to creatinine ratio (uACR) was 6.9 mg/mmol, follow-up was 4.3 years and 21% had the composite endpoint. On cross-validation in derivation (n = 686), KidneyIntelX had an AUC of 0.77 (95% CI 0.74, 0.79). In validation (n = 460), the AUC was 0.77 (95% CI 0.76, 0.79). By comparison, the AUC for the clinical model was 0.62 (95% CI 0.61, 0.63) in derivation and 0.61 (95% CI 0.60, 0.63) in validation. Using derivation cut-offs, KidneyIntelX stratified 46%, 37% and 17% of the validation cohort into low-, intermediate- and high-risk groups for the composite kidney endpoint, respectively. The PPV for progressive decline in kidney function in the high-risk group was 61% for KidneyIntelX vs 40% for the highest risk strata by KDIGO categorisation (p < 0.001). Only 10% of those scored as low risk by KidneyIntelX experienced progression (i.e., NPV of 90%). The NRIevent for the high-risk group was 41% (p < 0.05). Conclusions KidneyIntelX improved prediction of kidney outcomes over KDIGO and clinical models in individuals with early stages of DKD. Graphical abstract

show abstract

Section: Discussionmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

Derivation and validation of a machine learning risk score using biomarker and electronic patient data to predict progression of diabetic kidney disease

Chan

Nadkarni

Fleming³

et al. 2021

Diabetologia

Self Cite

111

View full text Add to dashboard Cite

show abstract

“…Chauhan et al 32 utilized an EHR dataset to develop a random forest-based prediction model to predict incident renal failure. EHR datasets have also been used to develop traditional statistical models, including risk scores for AF.…”

Section: Discussionmentioning

confidence: 99%

Cohort Design and Natural Language Processing to Reduce Bias in Electronic Health Records Research: The Community Care Cohort Project

Khurshid

Reeder

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Electronic health records (EHRs) promise to enable broad-ranging discovery with power exceeding that of conventional research cohort studies. However, research using EHR datasets may be subject to selection bias, which can be compounded by missing data, limiting the generalizability of derived insights. Methods: Mass General Brigham (MGB) is a large New England-based healthcare network comprising seven tertiary care and community hospitals with associated outpatient practices. Within an MGB-based EHR warehouse of >3.5 million individuals with at least one ambulatory care visit, we approximated a community-based cohort study by selectively sampling individuals longitudinally attending primary care practices between 2001-2018 (n=520,868), which we named the Community Care Cohort Project (C3PO). We also utilized pre-trained deep natural language processing (NLP) models to recover vital signs (i.e., height, weight, and blood pressure) from unstructured notes in the EHR. We assessed the validity of C3PO by deploying established risk models including the Pooled Cohort Equations (PCE) and the Cohorts for Aging and Genomic Epidemiology Atrial Fibrillation (CHARGE-AF) score, and compared model performance in C3PO to that observed within typical EHR Convenience Samples which included all individuals from the same parent EHR with sufficient data to calculate each score but without a requirement for longitudinal primary care. All analyses were facilitated by the JEDI Extractive Data Infrastructure pipeline which we designed to efficiently aggregate EHR data within a unified framework conducive to regular updates. Results: C3PO includes 520,868 individuals (mean age 48 years, 61% women, median follow-up 7.2 years, median primary care visits per individual 13). Estimated using reports, C3PO contains over 2.9 million electrocardiograms, 450,000 echocardiograms, 12,000 cardiac magnetic resonance images, and 75 million narrative notes. Using tabular data alone, 286,009 individuals (54.9%) had all vital signs available at baseline, which increased to 358,411 (68.8%) after NLP recovery (31% reduction in missingness). Among individuals with both NLP and tabular data available, NLP-extracted and tabular vital signs obtained on the same day were highly correlated (e.g., Pearson r range 0.95-0.99, p<0.01 for all). Both the PCE models (c-index range 0.724-0.770) and CHARGE-AF (c-index 0.782, 95% 0.777-0.787) demonstrated good discrimination. As compared to the Convenience Samples, AF and MI/stroke incidence rates in C3PO were lower and calibration error was smaller for both PCE (integrated calibration index range 0.012-0.030 vs. 0.028-0.046) and CHARGE-AF (0.028 vs. 0.036). Conclusions: Intentional sampling of individuals receiving regular ambulatory care and use of NLP to recover missing data have the potential to reduce bias in EHR research and maximize generalizability of insights.

show abstract

“…This score is then used to stratify patients into low, intermediate, or high risk of experiencing progression over five years. This diagnostic tool, validated elsewhere, improves the ability to accurately identify patients at high risk for adverse kidney events (61% classified as high risk with KidneyIntelX compared to 40% classified as high risk with standard of care KDIGO criteria (p<0.001), allows physicians to optimize the treatment pathway with preventative measures at an earlier stage, which can slow progression to ESRD and improve patient outcomes [17,21]. The KidneyIntelX test can be integrated with EHR systems and supporting Care Navigation teams and software of health Nephrology, to proactively identify patients that meet the intended use criteria and therefore may benefit from the KidneyIntelX test (T2DM with DKD stages 1-3).…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Payer budget impact of an artificial intelligence in vitro diagnostic to modify diabetic kidney disease progression

Datar

Burchenal

Donovan

et al. 2021

Journal of Medical Economics

Self Cite

View full text Add to dashboard Cite

To evaluate the U.S. payer budget-impact of KidneyIntelX, an artificial intelligence-enabled in vitro diagnostic to predict kidney function decline in Type 2 Diabetic Kidney Disease (T2DKD) patients, stages 1-3b. Materials and MethodsWe developed an Excel-based model according to International Society of Pharmacoeconomics and Outcomes Research (ISPOR) good practices to assess U.S. payer budget impact associated with use of the KidneyIntelX test to optimize therapy T2DKD patients compared to standard of care (SOC) (without KidneyIntelX). A hypothetical cohort of 100,000 stage 1-3b T2DKD patients was followed for five years. Peer-reviewed publications were used to identify model parameter estimates. KidneyIntelX costs incremental to SOC (without KidneyIntelX) included test cost, additional prescription medication use, specialist referrals and PCP office visits. Patients managed with KidneyIntelX experienced a 20% slowed progression rate compared to SOC (without KidneyIntelX) attributed to slowed DKD progression, delayed or prevented dialysis and transplants, and reduced dialysis crashes. Associated costs were compared to SOC (without KidneyIntelX). Sensitivity analyses were conducted by varying the definition of progression and the DKD progression rate associated with KidneyIntelX testing and related interventions. ResultsProjected undiscounted base case 5-year savings for 100,000 patients tested with KidneyIntelX were $1.052 billion, attributed mostly to slowed progression through DKD stages. The breakeven point for the health plan adopting KidneyIntelX is expected to occur prior to year two after adoption. Sensitivity analysis based on assessment of the most conservative definition of progression and a 5% reduction in progression rate attributed to KidneyIntelX, resulted in a projected 5-year savings of $145 million associated with KidneyIntelX. A c c e p t e d M a n u s c r i p t Limitations and ConclusionsLimitations included reliance on literature-based parameter estimates, including effect size of delayed progression supported by literature. Incorporating KidneyIntelX in contemporary care of early-stage T2DKD patients is projected to result in substantial savings to payers.

show abstract

Initial Validation of a Machine Learning-Derived Prognostic Test (KidneyIntelX) Integrating Biomarkers and Electronic Health Record Data To Predict Longitudinal Kidney Outcomes

Cited by 20 publications

References 38 publications

Derivation and validation of a machine learning risk score using biomarker and electronic patient data to predict progression of diabetic kidney disease

Derivation and validation of a machine learning risk score using biomarker and electronic patient data to predict progression of diabetic kidney disease

Cohort Design and Natural Language Processing to Reduce Bias in Electronic Health Records Research: The Community Care Cohort Project

Payer budget impact of an artificial intelligence in vitro diagnostic to modify diabetic kidney disease progression

Contact Info

Product

Resources

About