Recommendations for Determining the Validity of Consumer Wearables and Smartphones for the Estimation of Energy Expenditure: Expert Statement and Checklist of the INTERLIVE Network

Abstract

Background Consumer wearables and smartphone devices commonly offer an estimate of energy expenditure (EE) to assist in the objective monitoring of physical activity to the general population. Alongside consumers, healthcare professionals and researchers are seeking to utilise these devices for the monitoring of training and improving human health. However, the methods of validation and reporting of EE estimation in these devices lacks rigour, negatively impacting on the ability to make comparisons between devices and provide transparent accuracy. Objectives The Towards Intelligent Health and Well-Being Network of Physical Activity Assessment (INTERLIVE) is a joint European initiative of six universities and one industrial partner. The network was founded in 2019 and strives towards developing best-practice recommendations for evaluating the validity of consumer wearables and smartphones. This expert statement presents a best-practice validation protocol for consumer wearables and smartphones in the estimation of EE. Methods The recommendations were developed through (1) a systematic literature review; (2) an unstructured review of the wider literature discussing the potential factors that may introduce bias during validation studies; and (3) evidence-informed expert opinions from members of the INTERLIVE network. Results The systematic literature review process identified 1645 potential articles, of which 62 were deemed eligible for the final dataset. Based on these studies and the wider literature search, a validation framework is proposed encompassing six key domains for validation: the target population, criterion measure, index measure, testing conditions, data processing and the statistical analysis. Conclusions The INTERLIVE network recommends that the proposed protocol, and checklists provided, are used to standardise the testing and reporting of the validation of any consumer wearable or smartphone device to estimate EE. This in turn will maximise the potential utility of these technologies for clinicians, researchers, consumers, and manufacturers/developers, while ensuring transparency, comparability, and replicability in validation. Trial Registration PROSPERO ID: CRD42021223508.

Wearable activity trackers–advanced technology or advanced marketing?

“…Although insufficient validation studies exist for many of the features promoted by wearable technology, EE is one of the more frequently examined physiological outputs. In general, research demonstrates strong reliability for wearable technologies (Evenson et al 2015 ), but poor validity (Düking et al 2020 ; Fuller et al 2020 ; O’Driscoll et al 2020 ; Argent et al 2022 ). When pitted against the gold standard doubly label water technique, accelerometers demonstrated varying degrees of validity, prompting researchers to call for further development and evaluation of wearable technology (Plasqui and Westerterp 2007 ; Plasqui et al 2013 ; Murakami et al 2019 ).…”

Disagreements in physical activity monitor validation study guidelines create challenges in conducting validity studies

“…In recent years, multiple groups consisting of researchers and/or industry partners have developed and published guidelines/frameworks designed to be a helpful resource for those planning an activity monitor validation study (1)(2)(3)(4)(5)(6)(7)(8). However, problems arise for those planning validation studies when inconsistencies among these recommendation papers exist.…”

Validity of three smartwatches in estimating energy expenditure during outdoor walking and running

“…Estimating EE by wrist-worn wearable devices seems convenient, considering the linear relationship of heart rate and/or physical activity with gaseous exchange. Consequently, numerous validation studies on EE estimations have been conducted, but provided heterogeneous results ( O’driscoll et al, 2020 ; Argent et al, 2022 ). Overall, it appears that commercially available wrist- or arm-worn devices did not show a sufficient accuracy, while large between-device differences were observed ( O’driscoll et al, 2020 ).…”

“…The differences in the accuracy of different devices may be dependent on several factors. Most importantly, the accuracy of the input parameters, such as the accuracy of heart rate from photoplethysmography (PPG) sensors affects EE estimations ( O’driscoll et al, 2020 ; Argent et al, 2022 ). In this context, it appears that the accuracy of EE estimation may vary with the type and intensity of activities ( O’driscoll et al, 2020 ).…”

“…Among the most important variables appears to be the intended use of the device, i.e. validating EE estimations in settings that represent the actual use by consumers ( Argent et al, 2022 ). This is because specific sensors may be used for specific activities.…”