Evaluation of Machine Learning to Detect Influenza Using Wearable Sensor Data and Patient-Reported Symptoms: Cohort Study

Abstract

Background: Machine learning offers quantitative pattern recognition analysis of wearable device data and has the potential to detect illness onset and monitor influenza-like illness (ILI) in infected patients.Objective: To evaluate the ability of machine learning algorithms to distinguish between influenza-positive and influenzanegative participants in a cohort of symptomatic ILI patients using wearable sensor (activity) data and self-reported symptom data during the latent and early symptomatic period of ILI.Methods: This cohort study used the extreme gradient boosting (XGBoost) classifier to determine whether a participant was influenza positive or negative based on three models: symptoms data only; activity data only; and combined symptoms and activity data. Data were collected from the Home Testing of Respiratory Illness (HTRI) study and FluStudy2020, both conducted between December 2019 and October 2020. Analyses included participants in these studies with an at-home influenza diagnostic test result. Fitbit devices were used to measure participants' steps, heart rate, and sleep data. Participants detailed their ILI symptoms, healthcare-seeking behaviors, and quality of life. Model performance was assessed by: area under the curve (AUC), balanced accuracy, recall (sensitivity), precision (positive predictive value [PPV]), and F2 (weighted harmonic mean of precision and recall) score.Results: An influenza diagnostic test result was available for 953 and 925 participants in HTRI and FluStudy2020 respectively, of whom 848 and 840 had activity data. The highest-performing model used symptoms and activity data (test set means: 0.74 AUC, 0.68 balanced accuracy, 0.70 sensitivity, 0.38 PPV, 0.60 F2). The symptoms-only model had the second-best performance (0.74 AUC, 0.69 balanced accuracy, 0.65 sensitivity, 0.42 PPV, 0.58 F2). The top features guiding influenza detection were cough, mean resting heart rate during main sleep, and fever for the combined model, and cough, fever, and chills for the symptoms-only model. Conclusions:Machine learning algorithms had insufficient accuracy to detect influenza, suggesting that previous findings from research-grade sensors tested in highly controlled experimental settings may not easily translate with scalable commercial-grade sensors. In the future, more advanced wearable sensors may improve their performance in the early detection and discrimination of viral respiratory infections. Clinical Trial: NCT04245800