Assessment of laboratory and daily energy expenditure estimates from consumer multi-sensor physical activity monitors

Abstract

Wearable physical activity monitors are growing in popularity and provide the opportunity for large numbers of the public to self-monitor physical activity behaviours. The latest generation of these devices feature multiple sensors, ostensibly similar or even superior to advanced research instruments. However, little is known about the accuracy of their energy expenditure estimates. Here, we assessed their performance against criterion measurements in both controlled laboratory conditions (simulated activities of daily living and structured exercise) and over a 24 hour period in free-living conditions. Thirty men (n = 15) and women (n = 15) wore three multi-sensor consumer monitors (Microsoft Band, Apple Watch and Fitbit Charge HR), an accelerometry-only device as a comparison (Jawbone UP24) and validated research-grade multi-sensor devices (BodyMedia Core and individually calibrated Actiheart™). During discrete laboratory activities when compared against indirect calorimetry, the Apple Watch performed similarly to criterion measures. The Fitbit Charge HR was less consistent at measurement of discrete activities, but produced similar free-living estimates to the Apple Watch. Both these devices underestimated free-living energy expenditure (-394 kcal/d and -405 kcal/d, respectively; P<0.01). The multi-sensor Microsoft Band and accelerometry-only Jawbone UP24 devices underestimated most laboratory activities and substantially underestimated free-living expenditure (-1128 kcal/d and -998 kcal/d, respectively; P<0.01). None of the consumer devices were deemed equivalent to the reference method for daily energy expenditure. For all devices, there was a tendency for negative bias with greater daily energy expenditure. No consumer monitors performed as well as the research-grade devices although in some (but not all) cases, estimates were close to criterion measurements. Thus, whilst industry-led innovation has improved the accuracy of consumer monitors, these devices are not yet equivalent to the best research-grade devices or indeed equivalent to each other. We propose independent quality standards and/or accuracy ratings for consumer devices are required.

Determining the Reliability of Several Consumer-Based Physical Activity Monitors

“…Additionally, only a portion of the day was spent in a free-living setting in our study, there was likely smaller variability in the data collected subsequently causing lower correlation coefficients than seen in the work of Dontje et al [12]. However, recent works have illustrated that CPAM's underestimate PA in free-living settings [14] and that the variability of these estimations is not consistent between CPAM models [13]. Collectively, available research suggests high or moderately-high reliability for most CPAMs and most dependent variables tested in free-living settings, supporting the use of these CPAMs during field-based PA monitoring [12].…”

“…More recently, manufacturers have produced CPAMs which incorporate variables, such as heart rate or other variables (e.g., skin temperature), into their algorithms (e.g., Apple Watch and Fitbit Charge). Indeed, a recent study showed these multi-sensor CPAMs showed improved energy expenditure estimations compared to single-sensor CPAMs [13]. As multi-sensor CPAMs become more common, the reliability of their newer variables (e.g., heart rate) and the influence of newer variables on other preexisting variables (e.g., kcals) should be investigated as there is likely crosstalk between sensors.…”

Accuracy of Apple Watch Measurements for Heart Rate and Energy Expenditure in Patients With Cardiovascular Disease: Cross-Sectional Study (Preprint)

“…The participant number of 40 patients was determined based on a literature study of comparable studies [5]- [8], [11], [13]. In most of these validation studies of wrist-worn health-tracking devices particpant numbers ranged from 20 to 60 patients.…”

“…Validation studies have been done to evaluate accuracy of HR, EE and other measurements in healthy subjects for a variety of fitness trackers [5][6][7][8][9][10][11][12][13][14][15][16][17]. Boudreaux et al [7] tested eight devices for accuracy of HR and EE measurements on healthy subjects and found that HR accuracy from wearable devices differed at different exercise intensities with an increasing underestimation of HR at higher exercise intensities.…”

Nivel de actividad física y grado de discapacidad en pacientes con dolor lumbar crónico

“…La actividad física, definida como una actividad muscular que incrementa el gasto energético por encima de los valores de reposo (14), ha sido una variable de difícil medición debido a sus múltiples dimensiones y complejidad (15). Los cuestionarios para su evaluación se han convertido en una valiosa herramienta pues poseen buena relación costo-eficiencia que se explica por su alta flexibilidad en la obtención de la información y por su capacidad para estimar el gasto energético (16).…”