<p>Evaluations of Commercial Sleep Technologies for Objective Monitoring During Routine Sleeping Conditions</p>

Abstract

Purpose The commercial market is saturated with technologies that claim to collect proficient, free-living sleep measurements despite a severe lack of independent third-party evaluations. Therefore, the present study evaluated the accuracy of various commercial sleep technologies during in-home sleeping conditions. Materials and Methods Data collection spanned 98 separate nights of ad libitum sleep from five healthy adults. Prior to bedtime, participants utilized nine popular sleep devices while concurrently wearing a previously validated electroencephalography (EEG)-based device. Data collected from the commercial devices were extracted for later comparison against EEG to determine degrees of accuracy. Sleep and wake summary outcomes as well as sleep staging metrics were evaluated, where available, for each device. Results Total sleep time (TST), total wake time (TWT), and sleep efficiency (SE) were measured with greater accuracy (lower percent errors) and limited bias by Fitbit Ionic [mean absolute percent error, bias (95% confidence interval); TST: 9.90%, 0.25 (−0.11, 0.61); TWT: 25.64%, −0.17 (−0.28, −0.06); SE: 3.49%, 0.65 (−0.82, 2.12)] and Oura smart ring [TST: 7.39%, 0.19 (0.04, 0.35); TWT: 36.29%, −0.18 (−0.31, −0.04); SE: 5.42%, 1.66 (0.17, 3.15)], whereas all other devices demonstrated a propensity to over or underestimate at least one if not all of the aforementioned sleep metrics. No commercial sleep technology appeared to accurately quantify sleep stages. Conclusion Generally speaking, commercial sleep technologies displayed lower error and bias values when quantifying sleep/wake states as compared to sleep staging durations. Still, these findings revealed that there is a remarkably high degree of variability in the accuracy of commercial sleep technologies, which further emphasizes that continuous evaluations of newly developed sleep technologies are vital. End-users may then be able to determine more accurately which sleep device is most suited for their desired application(s).

Evaluation of a Low-Cost Commercial Actigraph and Its Potential Use in Detecting Cultural Variations in Physical Activity and Sleep

“…Mantua et al [ 16 ], for example, found no difference between a research-based actigraph and PSG in total-sleep-time and sleep efficiency (see also Lüdtke et al [ 17 ]), but as expected, actigraphy performed poorer on a more finely grained analysis of sleep stages. Similar conclusions have been recently drawn by others who evaluated the performance of commercial smartwatches against PSG, pointing out that although the consumer-grade wearable devices are not yet appropriate for quantifying sleep at high resolution or identifying sleep stages over the night, they still allow adequate estimates of sleep/wake distributions [ 18 , 19 ].…”

“…These problems can be surpassed with objective tracking of sleep and activity using affordable smartwatches. These devices also have the advantage that they can be used in low-income and developing countries and add to our knowledge of intercultural differences or similarities in sleep habits and activity patterns [ 18 ]. Yet, we emphasize that it is inevitable that such low-cost consumer devices are continuously monitored and evaluated by independent research in order to ensure reliable measurements and data, as well as identify limitations.…”

Trait-like nocturnal sleep behavior identified by combining wearable, phone-use, and self-report data

“…Furthermore, the relatively lower cost of consumer-grade wearable devices compared to research-grade actigraphy (or PSG) enhances the scalability of this approach. Recent validation studies have reported that the performance of the Oura ring for measurement of sleep timing and duration was comparable to that of research actigraphy 29 – 31 (see “Methods”), while Tappigraphy shows a high correlation with actigraphy 18 .…”

“…Overall, the accuracy of sleep–wake detection was good. Two studies found no systematic error in TST estimates, with only a small absolute error as compared to PSG (87.8% of nights within 30 min error) 30 and ambulatory EEG (7.39% mean absolute percentage error 31 ). Two other studies reported modest but significant overestimation of Oura-derived TST by about 15 min, compared to PSG 52 , and actigraphy 29 in adults, while another study reported substantial underestimation of TST compared to PSG, in an adolescent population (32–47 min) 53 .…”

The Promise of Sleep: A Multi-Sensor Approach for Accurate Sleep Stage Detection Using the Oura Ring

“…One of the main strengths of this work is the high sensitivity and specificity reported across all stages, ranging from 74% to 98%. While other studies have shown similar results for the detection of a specific stage, such as, e.g., deep sleep, this typically comes at the expense of the performance in detecting other sleep stages, e.g., resulting in REM or wake sensitivity as low as 50% [ 26 , 45 , 46 ]. A significant advance made by this report is the finding that, combining multiple sensor data streams from the finger, as well as circadian-features, and, finally, feature normalization, high sensitivity and specificity for all sleep stages can be accomplished.…”