Sleep Pose Recognition in an ICU Using Multimodal Data and Environmental Feedback

Abstract

Abstract. Clinical evidence suggests that sleep pose analysis can shed light onto patient recovery rates and responses to therapies. In this work, we introduce a formulation that combines features from multimodal data to classify human sleep poses in an Intensive Care Unit (ICU) environment. As opposed to the current methods that combine data from multiple sensors to generate a single feature, we extract features independently. We then use these features to estimate candidate labels and infer a pose. Our method uses modality trusts -each modality's classification ability -to handle variable scene conditions and to deal with sensor malfunctions. Specifically, we exploit shape and appearance features extracted from three sensor modalities: RGB, depth, and pressure. Classification results indicate that our method achieves 100 % accuracy (outperforming previous techniques by 6 %) in bright and clear (ideal) scenes, 70 % in poorly illuminated scenes, and 90 % in occluded ones.

Eye-CU: Sleep pose classification for healthcare using multimodal multiview data

“…Future analysis will seek to quantify and typify pose sequences (i.e., duration and transition). Future work [7] with 7%, [18] with 55%, and cc-LS with 86.7% for dark and occluded scenes. The matrices show the matches between estimated (l) and ground truth (l * ) indices.…”

“…The P M pM has the lowest performance of 76.7% using sh views of a dark and occluded scene. The method from[18] performs below 50% and the method from[7] is not suited for such conditions. The top row identifies the configuration.…”

Deep Eye-CU (DECU): Summarization of Patient Motion in the ICU

“…5) indicate that Inception features perform better than gMOMs, HOG, and VGG features. Parameters for gMOM and HOG extraction are obtained from [15]. Background subtraction and calibration procedures from [23] are applied prior to feature extraction.…”

“…Static pose classification in a range of simulated healthcare environments is addressed in [15], where the authors use modality trust and RGB, Depth, and Pressure data. In [16], the authors introduce a coupled-constrained optimization technique that allows them to remove the pressure sensor and increase pose classification performance.…”

A Multiview Multimodal System for Monitoring Patient Sleep

“…However, the system requires complex calibration and a top clear view of the patient's body configuration. Pose classification is also tackled in [38] using RGB, depth, and pressure sensors in simulated healthcare environments. The authors combine RGB, depth, and pressure modalities with room sensors to weight modality reliability.…”