External validation of a data‐driven algorithm for muscular activity identification during sleep

Abstract

Several automated methods for scoring periodic limb movements during sleep (PLMS) and rapid eye movement (REM) sleep without atonia (RSWA) have been proposed, but most of them were developed and validated on data recorded in the same clinic, thus they may be biased. This work aims to validate our data‐driven algorithm for muscular activity detection during sleep, originally developed based on data recorded and manually scored at the Danish Center for Sleep Medicine. The validation was carried out on a cohort of 240 participants, including de novo Parkinson's disease (PD) patients and neurologically healthy controls, whose sleep data were recorded and manually evaluated at Paracelsus‐Elena Klinik, Kassel, Germany. In the German cohort, the algorithm showed generally good agreement between manual and automated PLMS indices, and identified with 88.75% accuracy participants with PLMS index above 15 PLMS per hour of sleep, and with 84.17% accuracy patients suffering from REM sleep behaviour disorder (RBD) showing RSWA. By comparing the algorithm performances in the Danish and German cohorts, we hypothesized that inter‐clinical differences may exist in the way limb movements are manually scored and how healthy controls are defined. Finally, the algorithm performed worse in PD patients, probably as a result of increased artefacts caused by abnormal motor events related to neurodegeneration. Our algorithm can identify, with reasonable performance, participants with RBD and increased PLMS index from data recorded in different centres, and its application may reveal inter clinical differences, which can be overcome in the future by applying automated methods.

Current Concepts and Controversies in the Management of REM Sleep Behavior Disorder

“…Several studies have now shown that addition of upper-extremity electromyography significantly improves the sensitivity for RBD diagnosis; however, rare patients present with movements largely limited to the legs, highlighting the importance of evaluating multiple muscles for RSWA as well as a video review of PSG for abnormal behaviors [31,40]. More recently, automated and machine learning approaches for RSWA scoring have been published, which have shown promise for more time-efficient and standardized approaches for RBD diagnosis, although these approaches require further validation in large-scale multicenter studies [41,42]. Standardization of RSWA scoring and polysomnography recording practices to optimize RBD detection will be important for the accurate recruitment of patients and generalization of findings from therapeutic trials in RBD.…”

Automated Movement Detection with Dirichlet Process Mixture Models and Electromyography

“…Each participant was given a PLMS-index score which details the average number of limb movements per hour of sleep. Using the aforementioned techniques [15], [17], this study demonstrated an automated classification of participants with PLMD and RBD with an accuracy of 88.75% and 84.17%, respectively [16]. Once more this study was able to assess the performance of limb movement detection by achieving an automated PLMS-index score that correlated to the manual score by 84.99% and only had a slight bias towards over-predicting the PLMS-index [16].…”

“…As a result the concept of automated movement detection algorithm lends itself towards a non-parametric model that can incorporate numerous sensors and compensate for movement which can vary greatly in magnitude and severity. A handful of other studies demonstrate this through limb movement detection in participants with RBD and PLMD [15]- [17].…”

“…From a dataset containing 27 healthy controls and 36 participants diagnosed with PLMD, this study was able to utilise this semi-supervised approach to detect PLMD participants with 82%. As an extension of this study, , applied this technique on a mixed cohort of 27 healthy controls, 36 individuals diagnosed with PLMD, and 29 participants diagnosed with RBD [17]. While these studies didn't explore the performance of limb-movement detection (as manual annotations of limb movement are rare), it did validate its utility in distinguishing RBD and PLMD participants from healthy individuals.…”

Improved automatic identification of isolated rapid eye movement sleep behavior disorder with a 3D time‐of‐flight camera

“…Future development of 3D video analysis in the context of RBD should aim at expanding the use of this technology in home settings without focusing on REM sleep only and without the need for manually selecting upper and lower body ROIs. Previous studies showing that people with iRBD have increased muscular activity also in NREM sleep [ 9 , 30 , 31 ] would support the hypothesis that identification of REM sleep might not be necessary, but further studies are needed to confirm this hypothesis. Furthermore, future studies should evaluate whether upper and lower ROIs can be automatically selected with segmentation [ 32 ] and pose estimation algorithms [ 33 ].…”