Reliability of Sleep Measures from Four Personal Health Monitoring Devices Compared to Research-Based Actigraphy and Polysomnography

Abstract

Polysomnography (PSG) is the “gold standard” for monitoring sleep. Alternatives to PSG are of interest for clinical, research, and personal use. Wrist-worn actigraph devices have been utilized in research settings for measures of sleep for over two decades. Whether sleep measures from commercially available devices are similarly valid is unknown. We sought to determine the validity of five wearable devices: Basis Health Tracker, Misfit Shine, Fitbit Flex, Withings Pulse O2, and a research-based actigraph, Actiwatch Spectrum. We used Wilcoxon Signed Rank tests to assess differences between devices relative to PSG and correlational analysis to assess the strength of the relationship. Data loss was greatest for Fitbit and Misfit. For all devices, we found no difference and strong correlation of total sleep time with PSG. Sleep efficiency differed from PSG for Withings, Misfit, Fitbit, and Basis, while Actiwatch mean values did not differ from that of PSG. Only mean values of sleep efficiency (time asleep/time in bed) from Actiwatch correlated with PSG, yet this correlation was weak. Light sleep time differed from PSG (nREM1 + nREM2) for all devices. Measures of Deep sleep time did not differ from PSG (SWS + REM) for Basis. These results reveal the current strengths and limitations in sleep estimates produced by personal health monitoring devices and point to a need for future development.

Utility of the Fitbit Flex to evaluate sleep in major depressive disorder: A comparison against polysomnography and wrist-worn actigraphy

“…Although not many validation studies have been performed on fitness activity trackers in the domain of sleep, this demonstration of an overestimation for total sleep time and inability to accurately identify wake epochs has been corroborated by studies on women with insomnia (de Zambotti et al, 2015) and adolescents referred for clinical PSG (Meltzer et al, 2015). Contrary to the results of these investigations, one study in healthy, young adults demonstrated comparable results in estimation of total sleep time for multiple fitness activity trackers, relative to PSG (Mantua et al, 2016). However, because epoch-by-epoch comparisons were not performed, the full performance characteristics of the fitness trackers utilized in this study, relative to PSG, could not be determined.…”

Physical Activity and Sleep Patterns in Hemodialysis Patients in a Suburban Environment

“…The Fitbit® Flex™ can be used to measure sleep, as long as the user is aware that the data collected point to an overestimation of sleep duration and quality. Consequently, our measurements of sleep duration and quality, which are already low, may have been overestimated [15]. …”

Noncontact Pressure-Based Sleep/Wake Discrimination

“…A comparison of these and similar commercial personal health monitoring devices against global sleep measures (via PSG) in healthy populations has found no significant differences between them in terms of the their ability to estimate total sleep time [41]. However, it is reasonable to assume that a simple in/out of bed sensor may provide similar accuracy levels in terms of total sleep time.…”

“…An epoch-by-epoch comparison is necessary to provide a more rigorous evaluation of the performance of these systems. Furthermore, healthy adult users were found to be unreliable for any devices which required the user to initiate a sleep tracking setting [41]. Therefore, when moving to long-term sleep monitoring, particularly in populations with potential memory issues, an ambient technology requiring no user interaction is desirable.…”