Sleep posture classification with multi-stream CNN using vertical distance map

Sleep quality is known to have a considerable impact on human health. Recent research shows that head and body pose play a vital role in affecting sleep quality. This paper presents a deep multi-task learning network to perform head and upper-body detection and pose classification during sleep. The proposed system has two major advantages: first, it detects and predicts upper-body pose and head pose simultaneously during sleep, and second, it is a contact-free home security camera-based monitoring system that can work on remote subjects, as it uses images captured by a home security camera. In addition, a synopsis of sleep postures is provided for analysis and diagnosis of sleep patterns. Experimental results show that our multi-task model achieves an average of 92.5% accuracy on challenging datasets, yields the best performance compared to the other methods, and obtains 91.7% accuracy on the real-life overnight sleep data. The proposed system can be applied reliably to extensive public sleep data with various covering conditions and is robust to real-life overnight sleep data.

show abstract

“…Some studies use depth cameras [16][17][18][19]. Grimm et al [16] use depth maps to detect sleep posture.…”

Section: Sleep Posture Classificationmentioning

confidence: 99%

A Vision-Based System for In-Sleep Upper-Body and Head Pose Classification

Wang

Hung

2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…A multi-stream CNN was also applied in this field, which was based on depth images for identifying ten sleep postures with high accuracy [14]. A sleep monitoring system by embedding radio frequency identification (RFID) tags was proposed for sleeping posture recognition and body movement detection, which used a convolutional neural network (CNN) to identify the sleeping postures.…”

Section: A Sleep Posture Recognitionmentioning

confidence: 99%

“…However, this kind of sleeping posture is easy to lead to the fall of the tongue root and block breathing, which is not suitable for people who often snore or have respiratory diseases. Lying on the right side is conducive to the normal operation of the gastrointestinal tract and will not compress the heart, but it can affect the movement of the right lung [2]. On the other hand, monitoring the sleep stage is also an important way to evaluate sleep state.…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Learning From Multi-Sensor Data for Sleep Recognition

2020

View full text Add to dashboard Cite

Sleep recognition refers to detection or identification of sleep posture, state or stage, which can provide critical information for the diagnosis of sleep diseases. Most of sleep recognition methods are limited to single-task recognition, which only involves single-modal sleep data, and there is no generalized model for multi-task recognition on multi-sensor sleep data. Moreover, the shortage and imbalance of sleep samples also limits the expansion of the existing machine learning methods like support vector machine, decision tree and convolutional neural network, which lead to the decline of the learning ability and overfitting. Self-supervised learning technologies have shown their capabilities to learn significant feature representations. In this paper, a novel self-supervised learning model is proposed for sleep recognition, which is composed of an upstream self-supervised pre-training task and a downstream recognition task. The upstream task is conducted to increase the data capacity, and the information of frequency domain and the rotation view are used to learn the multi-dimensional sleep feature representations. The downstream task is undertaken to fuse bidirectional long-short term memory and conditional random field as the sequential data recognizer to produce the sleep labels. Our experiments shows that our proposed algorithm provide promising results in sleep identification and can further be applied in clinical and smart home environments as a diagnostic tool. The source code is provided at: ''https://github.com/zhaoaite/SSRM ''.

show abstract

“…It's possible to find many other recent works in the literature that use CNN in image classification in different tasks (eg. : [25], [26] and [27]).…”

Section: Introductionmentioning

confidence: 99%

Road Surface Classification with Images Captured From Low-cost Camera - Road Traversing Knowledge (RTK) Dataset

Rateke

Justen

Wangenheim

2019

RITA

View full text Add to dashboard Cite

The type of road pavement directly influences the way vehicles are driven. It’s common to find papers that deal with path detection but don’t take into account major changes in road surface patterns. The quality of the road surface has a direct impact on the comfort and especially on the safety of road users. In emerging countries it’s common to find unpaved roads or roads with no maintenance. Unpaved or damaged roads also impact in higher fuel costs and vehicle maintenance. This kind of analysis can be useful for both road maintenance departments as well as for autonomous vehicle navigation systems to verify potential critical points. For the experiments accomplishment upon the surface types and quality classification, we present a new dataset, collected with a low-cost camera. This dataset has examples of good and bad asphalt (with potholes and other damages) other types of pavement and also many examples of unpaved roads (with and without potholes). We also provide several frames from our dataset manually sorted in surface types for tests accuracy verification. Our road type and quality classifier was done through a simple Convolutional Neural Network with few steps and presents promising results in different datasets.

show abstract

Sleep posture classification with multi-stream CNN using vertical distance map

Cited by 19 publications

References 13 publications

A Vision-Based System for In-Sleep Upper-Body and Head Pose Classification

A Vision-Based System for In-Sleep Upper-Body and Head Pose Classification

Self-Supervised Learning From Multi-Sensor Data for Sleep Recognition

Road Surface Classification with Images Captured From Low-cost Camera - Road Traversing Knowledge (RTK) Dataset

Contact Info

Product

Resources

About