A machine learning-based phenotype for long COVID in children: an EHR-based study from the RECOVER program

Abstract

Background As clinical understanding of pediatric Post-Acute Sequelae of SARS CoV-2 (PASC) develops, and hence the clinical definition evolves, it is desirable to have a method to reliably identify patients who are likely to have post-acute sequelae of SARS CoV-2 (PASC) in health systems data. Methods and Findings In this study, we developed and validated a machine learning algorithm to classify which patients have PASC (distinguishing between Multisystem Inflammatory Syndrome in Children (MIS-C) and non-MIS-C variants) from a cohort of patients with positive SARS- CoV-2 test results in pediatric health systems within the PEDSnet EHR network. Patient features included in the model were selected from conditions, procedures, performance of diagnostic testing, and medications using a tree-based scan statistic approach. We used an XGboost model, with hyperparameters selected through cross-validated grid search, and model performance was assessed using 5-fold cross-validation. Model predictions and feature importance were evaluated using Shapley Additive exPlanation (SHAP) values. Conclusions The model provides a tool for identifying patients with PASC and an approach to characterizing PASC using diagnosis, medication, laboratory, and procedure features in health systems data. Using appropriate threshold settings, the model can be used to identify PASC patients in health systems data at higher precision for inclusion in studies or at higher recall in screening for clinical trials, especially in settings where PASC diagnosis codes are used less frequently or less reliably. Analysis of how specific features contribute to the classification process may assist in gaining a better understanding of features that are associated with PASC diagnoses.

De-black-boxing health AI: demonstrating reproducible machine learning computable phenotypes using the N3C-RECOVER Long COVID model in the All of Us data repository

“…We fully recognize the burden of Long COVID on the pediatric population-however, Long COVID appears to present differently in children, and these distinctions likely necessitate one or more separate models. 16 We should note that this exercise in ML-based phenotype reuse is not, and was not intended to be a validation of the accuracy of the phenotype. ML-based computable phenotypes present challenges with performance assessment, 17 and the challenge is even greater in the case of a new disease like Long COVID, where few concrete diagnostic guidelines or gold standards exist.…”

Clinical-based phenotypes in children with pediatric post-COVID-19 condition

“…Validation of these cluster analyses in a larger population is recommended to increase generalizability. Machine learning-based clustering has previously been applied to identify potential PPCC patients based on their clinical records [40], and has also been employed in the identification of PCC phenotypes in adults [30]. However, this method has not yet been performed for PPCC cluster analyses, making it a potentially promising tool.…”

EHR-based Case Identification of Pediatric Long COVID: A Report from the RECOVER EHR Cohort

“…Our two-pronged approach to identifying Long COVID using clinician chart review and a computable phenotype is a strength of the study as previous research that used diagnosis codes or machine learning algorithms did not incorporate a review of patient charts [19][20] . By incorporating both methods, we were able to qualitatively review cases of discordance.…”

“…To improve identification of patients with Long COVID, computable phenotyping techniques, which involve developing a set of rules to identify patients with a disorder, have been used in Long COVID studies. Long COVID phenotypes for adult 19 and pediatric 20 patients have been developed using machine-learning approaches that leverage large numbers of clinical features. For example, in a recent pediatric study, a machine learning algorithm demonstrated high precision in classifying both general and MIS-C-specific forms of PASC, with recall rates of up to 70% 20 .…”

“…Long COVID phenotypes for adult 19 and pediatric 20 patients have been developed using machine-learning approaches that leverage large numbers of clinical features. For example, in a recent pediatric study, a machine learning algorithm demonstrated high precision in classifying both general and MIS-C-specific forms of PASC, with recall rates of up to 70% 20 . Training these supervised learning models requires a labeled cohort of patients who likely have Long COVID based on healthcare utilization or Long COVID diagnosis codes.…”