Evaluation of a machine learning approach utilizing wearable data for prediction of SARS-CoV-2 infection in healthcare workers

Abstract

Objective To determine whether a machine learning model can detect SARS-CoV-2 infection from physiological metrics collected from wearable devices. Materials and Methods Health care workers from seven hospitals were enrolled and prospectively followed in a multicenter observational study. Subjects downloaded a custom smart phone app and wore Apple Watches for the duration of the study period. Daily surveys related to symptoms and the diagnosis of COVID-19 were answered in the app. Results We enrolled 407 participants with 49 (12%) having a positive nasal SARS-CoV-2 polymerase chain reaction test during follow-up. We examined five machine-learning approaches and found that gradient-boosting machines (GBM) had the most favorable validation performance. Across all testing sets, our GBM model predicted SARS-CoV-2 infection with an average area under the receiver operating characteristic (auROC) = 86.4% (Confidence Interval 84-89%). The model was calibrated to value sensitivity over specificity, achieving an average sensitivity of 82% (CI ±∼4%) and specificity of 77% (CI ±∼1%). The most important predictors included parameters describing the circadian heart rate variability mean (MESOR) and peak-timing (acrophase), and age. Discussion We show that a tree-based ML algorithm applied to physiological metrics passively collected from a wearable device can identify and predict SARS-CoV-2 infection. Conclusion Applying machine learning models to the passively collected physiological metrics from wearable devices may improve SARS-CoV-2 screening methods and infection tracking. Lay Summary The goal of the study is to determine if SARS-CoV-2 infections, which cause Coronavirus Disease 2019 (COVID-19), can be detected using machine learning algorithms applied to the information collected by wearable devices. Four hundred and nine health care workers were enrolled from 7 hospitals in New York City. Participants downloaded a custom smart phone application and were provided with an Apple Watch, if they did not have one of their own. Daily questions collected information from participants about how they feel and whether they were diagnosed with COVID-19. We found that a type of machine learning algorithm, called gradient boosting machines was able to reliably predict SARS-CoV-2 infections by combining various metrics collected from the Apple Watch. We found markers of heart rate variability, or the calculation of the small-time differences between each heartbeat, to be important in identifying infections. These findings demonstrate that wearable devices may improve screening for SARS-CoV-2 infections and the overall tracking of infections.

The Impact of SARS-CoV-2 Infection on Heart Rate Variability: A Systematic Review of Observational Studies with Control Groups

“…The inter-rater agreement rate among the researchers in the full texts review was 97.67% (42/43). Therefore, 17 observational studies (total participants, N = 3628) [49][50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65], including a study from other sources [61], were included in this review (Figure 1).…”

“…A total of four study types were analyzed, including four case-control studies [49,55,59,65], six retrospective analyses [54,[56][57][58]61,63], three crosssectional studies [51,52,60], and four prospective cohort studies [50,53,62,64]. Eleven studies [49,[51][52][53]55,[57][58][59]61,62,65] included SARS-CoV-2 negative individuals or healthy…”

“…Of the seventeen included studies, three [49,56,57] were conference abstracts, and fourteen [50][51][52][53][54][55][58][59][60][61][62][63][64][65] were journal articles. Nine studies (52.94%) were conducted in the United States [52][53][54][56][57][58]62,63,65], two (11.76%) were conducted in Turkey [49,61], and the remaining studies were conducted in China [60], India [55], Italy [51], Serbia [59], Spain [50] and Switzerland [64]. A total of four study types were analyzed, including four case-control studies [49,55,59,65], six retrospective analyses [54,[56][57][58]61,63], three cross-sectional studies [51,52,60], and four prospective cohort studies…”

“…Nine studies (52.94%) were conducted in the United States [52][53][54][56][57][58]62,63,65], two (11.76%) were conducted in Turkey [49,61], and the remaining studies were conducted in China [60], India [55], Italy [51], Serbia [59], Spain [50] and Switzerland [64]. A total of four study types were analyzed, including four case-control studies [49,55,59,65], six retrospective analyses [54,[56][57][58]61,63], three cross-sectional studies [51,52,60], and four prospective cohort studies [50,53,62,64]. Eleven studies [49,[51][52][53]55,[57][58][59]61,62,65] included SARS-CoV-2 negative individuals or healthy controls as the control group.…”

Smart Consumer Wearables as Digital Diagnostic Tools: A Review

“…Weiting et al used several algorithms, including SVM, RF, and naive Bayes, to build an ML algorithm ensemble for the purpose of predicting the cardiovascular risk from wearable healthcare data-collection devices [ 107 ]. In a study involving 407 participants using smartwatches, a gradient-boosting algorithm identified and predicted SARS-CoV2 infections [ 108 ]. Researchers have used multiple-instance learning via an embedded instance selection (MILES) method for feature transformation to detect obstructive cardiomyopathy [ 109 ].…”