Accuracy in Wrist-Worn, Sensor-Based Measurements of Heart Rate and Energy Expenditure in a Diverse Cohort

Abstract

The ability to measure physical activity through wrist-worn devices provides an opportunity for cardiovascular medicine. However, the accuracy of commercial devices is largely unknown. The aim of this work is to assess the accuracy of seven commercially available wrist-worn devices in estimating heart rate (HR) and energy expenditure (EE) and to propose a wearable sensor evaluation framework. We evaluated the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn, and Samsung Gear S2. Participants wore devices while being simultaneously assessed with continuous telemetry and indirect calorimetry while sitting, walking, running, and cycling. Sixty volunteers (29 male, 31 female, age 38 ± 11 years) of diverse age, height, weight, skin tone, and fitness level were selected. Error in HR and EE was computed for each subject/device/activity combination. Devices reported the lowest error for cycling and the highest for walking. Device error was higher for males, greater body mass index, darker skin tone, and walking. Six of the devices achieved a median error for HR below 5% during cycling. No device achieved an error in EE below 20 percent. The Apple Watch achieved the lowest overall error in both HR and EE, while the Samsung Gear S2 reported the highest. In conclusion, most wrist-worn devices adequately measure HR in laboratory-based activities, but poorly estimate EE, suggesting caution in the use of EE measurements as part of health improvement programs. We propose reference standards for the validation of consumer health devices ().

Stressing the accuracy: Wrist‐worn wearable sensor validation over different conditions

“…When residuals were not normally distributed, the goodness of fit of each of the following distribution families was assessed: log‐normal, gamma, beta, and Weibull. Since this final step of analyses was exploratory, with no clear hypotheses on predictors’ importance except some evidence from Shcherbina et al (), all possible combinations of selected predictors were compared based on the Akaike information criterion (AIC; i.e., comparing two or more models in terms of likelihood and parsimony) and the AIC weight (i.e., interpreted as the probability that a given model is better than other included models). Lower AICs and higher weights indicate a better fit (Akaike, ; Wagenmakers & Farrell, ).…”

“…Finally, our third aim was to explore which potential confounding variables may affect the measurement error. For instance, Shcherbina et al (2017) assessed the accuracy of several wearable wristbands in measuring average HR, finding higher errors for males, higher body mass index (BMI), darker skins, and movement conditions. The identification of these variables might help scholars decide when and how to use the device and which factors may affect their data quality.…”

Moving beyond self‐report data collection in the natural environment: A review of the past and future directions for ambulatory assessment in eating disorders

“…Regarding ambulatory assessment of cardiac functioning, there are a range of wristworn devices that allow for more convenient monitoring compared to the holter monitoring approach, which is generally considered a "gold standard" method (e.g., Akintola, van de Pol, Bimmel, Maan, & van Heemst, 2016). However, studies assessing the validity and reliability of newer wrist-worn heart monitoring devices are limited and inconclusive (e.g., Benedetto et al, 2018;Shcherbina et al, 2017;Thiebaud et al, 2018), which may be due in part to methodological variability (Sartor, Papini, Cox, & Cleland, 2018). This evidence therefore highlights the need for future research in this area.…”

Real-Time Monitoring in Home-Based Cardiac Rehabilitation Using Wrist-Worn Heart Rate Devices

“…However recently, a new generation of smart-watches and wrist-worn devices has improved the quality of measures in heart rate (HR), achieving a median error below 5% in laboratory-based activities [30]. Moreover, smart-watches and wrist-worn devices are expected to be a boon to mHealth technologies in physical activity sensing thanks to the recent tools and operating systems which enable application development [31].…”