A Vision-Based System for In-Sleep Upper-Body and Head Pose Classification

Abstract

Sleep quality is known to have a considerable impact on human health. Recent research shows that head and body pose play a vital role in affecting sleep quality. This paper presents a deep multi-task learning network to perform head and upper-body detection and pose classification during sleep. The proposed system has two major advantages: first, it detects and predicts upper-body pose and head pose simultaneously during sleep, and second, it is a contact-free home security camera-based monitoring system that can work on remote subjects, as it uses images captured by a home security camera. In addition, a synopsis of sleep postures is provided for analysis and diagnosis of sleep patterns. Experimental results show that our multi-task model achieves an average of 92.5% accuracy on challenging datasets, yields the best performance compared to the other methods, and obtains 91.7% accuracy on the real-life overnight sleep data. The proposed system can be applied reliably to extensive public sleep data with various covering conditions and is robust to real-life overnight sleep data.

Vision-based detection and quantification of maternal sleeping position in the third trimester of pregnancy in the home setting–Building the dataset and model

“…Over the last few years, several vision-based sleeping posture classification studies have been published including Grimm et al (2016) [ 90 ], Liu et al (2017) [ 91 ], Mohammadi et al (2018) [ 92 ], Wang et al (2019) [ 93 ], Li et al (2022) [ 94 ], and Akbarian et al (2019) [ 54 ]. Like Mohammadi et al [ 92 ] and Li et al [ 94 ], we used video from a readily available home-security camera, simulated sleeping position data, and incorporated the presence and absence of bed sheets.…”

“…Over the last few years, several vision-based sleeping posture classification studies have been published including Grimm et al (2016) [ 90 ], Liu et al (2017) [ 91 ], Mohammadi et al (2018) [ 92 ], Wang et al (2019) [ 93 ], Li et al (2022) [ 94 ], and Akbarian et al (2019) [ 54 ]. Like Mohammadi et al [ 92 ] and Li et al [ 94 ], we used video from a readily available home-security camera, simulated sleeping position data, and incorporated the presence and absence of bed sheets. Mohammadi et al [ 92 ] used a CNN algorithm (as we did), trained and validated their model using approximately 10,000 images, and completed resampling via leave-one-out (LOO) cross-validation (one participant was “held out”, and a model was trained on the remaining participants, and this was repeated for each participant).…”

“…We were able to calculate the average recall (“average sensitivity”) from Mohammadi et al’s testing data as 0.64 (SD 0.10) and 0.79 (SD 0.09) with and without bed sheets, respectively, which is comparable to the average recall across our six models at 0.67 (0.08) and 0.76 (SD 0.09) with and without bed sheets, respectively (see Fig 4 ). Comparison of Li et al’s [ 94 ] testing performance to ours is not possible because they only present classification accuracy (not recall, precision, nor mAP) but a direct comparison of accuracy is not possible because we cannot compute it for our results–accuracy computation requires the number of true negatives [ 95 ], which is not computed by YOLO, which is a detector and does not generate negative predictions. Furthermore, standard research and clinical scoring of body position is limited to left, right, supine, prone, or upright [ 96 ]; however, for physiologic reasons unique to pregnancy, our models score body position at a higher resolution by accounting for body twist and hybrid positions (e.g., supine thorax with left pelvic tilt).…”

Computer vision for automated seizure detection and classification: A systematic review

“…76 Apnea detection is described later with neonatal epilepsy, 29 and has a richer literature in sleep medicine. 77…”

“…CV methods have been used to detect gelastic (laughing) and dacrystic (crying) seizures with an accuracy of >98%, although with a very small dataset ( n = 4) consistent with their rare presentation 76 . Apnea detection is described later with neonatal epilepsy, 29 and has a richer literature in sleep medicine 77 …”