Graph-Based Denoising for Respiration and Heart Rate Estimation During Sleep in Thermal Video

Yang, Cheng; Hu, Menghan; Zhai, Guangtao; Zhang, Xiaoping

doi:10.1109/jiot.2022.3150147

Cited by 9 publications

(6 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Secondly, the use of a fully convolutional-based method which does not require higher-precision signal analysis algorithms allows for its deployment on machine learning accelerators [ 44 ] supporting such topologies, thus making the model suitable for smart home solutions [ 45 ], driver monitoring systems in L2+ Advanced Driver Assistance Systems (ADAS) [ 46 ], surveillance or security solutions [ 47 ], other embedded edge use cases, and potentially for other vital signs as well [ 48 ]. Lastly, previous methods were able to achieve satisfactory performance when additional pre-processing algorithms were applied, i.e., super-resolution [ 11 , 41 ], motion magnification [ 49 ], denoising [ 24 ], and others. This is caused by the fact that thermal imagery usually has poor resolution compared to visible light data, and thus images have to be enhanced before estimating vital signs.…”

Section: Discussionmentioning

confidence: 99%

“…Some methods are proposed to introduce the additional pre-processing step for improving the quality of input sequences, thus enhancing the dynamics of the pixel color changes used for vital sign extraction. These methods are based on color magnification [ 23 ], super-resolution—including Convolutional Neural Network (CNN)-based DRESNet [ 11 ] and Transformer-based TTSR [ 5 ], and denoising [ 24 ] with computer vision and deep neural network approaches. Studies [ 25 , 26 ] show examples of such a multi-step procedure.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth

Szankin,

Kwasniewska,

Ruminski

2023

J. Imaging

View full text Add to dashboard Cite

As healthcare costs continue to rise, finding affordable and non-invasive ways to monitor vital signs is increasingly important. One of the key metrics for assessing overall health and identifying potential issues early on is respiratory rate (RR). Most of the existing methods require multiple steps that consist of image and signal processing. This might be difficult to deploy on edge devices that often do not have specialized digital signal processors (DSP). Therefore, the goal of this study is to develop a single neural network realizing the entire process of RR estimation in a single forward pass. The proposed solution builds on recent advances in video recognition, capturing both spatial and temporal information in a multi-path network. Both paths process the data at different sampling rates to capture rapid and slow changes that are associated with differences in the temperature of the nostril area during the breathing episodes. The preliminary results show that the introduced end-to-end solution achieves better performance compared to state-of-the-art methods, without requiring additional pre/post-processing steps and signal-processing techniques. In addition, the presented results demonstrate its robustness on low-resolution thermal video sequences that are often used at the embedded edge due to the size and power constraints of such systems. Taking that into account, the proposed approach has the potential for efficient and convenient respiratory rate estimation across various markets in solutions deployed locally, close to end users.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth

Szankin,

Kwasniewska,

Ruminski

2023

J. Imaging

View full text Add to dashboard Cite

show abstract

“…The measured performance improves immunity to the subject skin color or the illumination, which provides the potential to monitor sleep during the night. [94] In comparison with other optical imaging approaches, hyperspectral imaging is a more advanced technique that can distinguish identification ability and contain rich information, resulting in high spatial and spectral resolution. The Microsoft Kinect sensor designed for gaming purposes brought a new way for physiological activity detection by capturing depth maps using time of flight technology.…”

Section: Advantages and Disadvantages Of Current Optical Approachesmentioning

confidence: 99%

“…The measured performance improves immunity to the subject skin color or the illumination, which provides the potential to monitor sleep during the night. [ 94 ]…”

Section: Advantages and Disadvantages Of Current Optical Approachesmentioning

confidence: 99%

Recent Progress of Optical Imaging Approaches for Noncontact Physiological Signal Measurement: A Review

Zhang

et al. 2023

Advanced Intelligent Systems

Self Cite

View full text Add to dashboard Cite

In recent years, optical imaging techniques have gained wide recognition for the measurement of vital signals, such as heart rate, respiratory rate, oxygen saturation, and blood pressure, which are crucial indicators for evaluating human health conditions in clinical examinations. There is a wide range of optical imaging methods for remote physiological signal monitoring, including RGB imaging, thermal imaging, hyperspectral imaging, depth imaging, and multimodal imaging, which provide spatial information compared to other noncontact measurement approaches, thereby allowing extensive applications in this area. In this survey, some fundamental knowledge about optical imaging methods for vital signal measurement is reviewed, including principles of various optical imaging techniques, processing methods for data analysis, discussion on advantages and disadvantages, application summary, and future prospects. This is a comprehensive overview of the noncontact physiological signal measurement of optical imaging approaches.

show abstract

“…Similarly, [19] introduced a semi-supervised learning framework with a high-order regularizer y L P y (P denotes 1-to-P-hop neighbor) as the graph signal prior. For a graphbased classification task, an iterative process is generally applied by alternately learning an aforementioned graph structure that fits the data and a graph-based classifier [16,[20][21][22][23][24][25], where the output of the iterative process is the predicted data labels.…”

Section: Introductionmentioning

confidence: 99%

MSGL+: Fast and Reliable Model Selection-Inspired Graph Metric Learning

Yang,

Zheng,

Zou

et al. 2023

Electronics

Self Cite

View full text Add to dashboard Cite

The problem of learning graph-based data structures from data has attracted considerable attention in the past decade. Different types of data can be used to infer the graph structure, such as graphical Lasso, which is learned from multiple graph signals or graph metric learning based on node features. However, most existing methods that use node features to learn the graph face difficulties when the label signals of the data are incomplete. In particular, the pair-wise distance metric learning problem becomes intractable as the dimensionality of the node features increases. To address this challenge, we propose a novel method called MSGL+. MSGL+ is inspired from model selection, leverages recent advancements in graph spectral signal processing (GSP), and offers several key innovations: (1) Polynomial Interpretation: We use a polynomial function of a certain order on the graph Laplacian to represent the inverse covariance matrix of the graph nodes to rigorously formulate an optimization problem. (2) Convex Formulation: We formulate a convex optimization objective with a cone constraint that optimizes the coefficients of the polynomial, which makes our approach efficient. (3) Linear Constraints: We convert the cone constraint of the objective to a set of linear ones to further ensure the efficiency of our method. (4) Optimization Objective: We explore the properties of these linear constraints within the optimization objective, avoiding sub-optimal results by the removal of the box constraints on the optimization variables, and successfully further reduce the number of variables compared to our preliminary work, MSGL. (5) Efficient Solution: We solve the objective using the efficient linear-program-based Frank–Wolfe algorithm. Application examples, including binary classification, multi-class classification, binary image denoising, and time-series analysis, demonstrate that MSGL+ achieves competitive accuracy performance with a significant speed advantage compared to existing graphical Lasso and feature-based graph learning methods.

show abstract

Graph-Based Denoising for Respiration and Heart Rate Estimation During Sleep in Thermal Video

Cited by 9 publications

References 72 publications

Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth

Thermal Image Processing for Respiratory Estimation from Cubical Data with Expandable Depth

Recent Progress of Optical Imaging Approaches for Noncontact Physiological Signal Measurement: A Review

MSGL+: Fast and Reliable Model Selection-Inspired Graph Metric Learning

Contact Info

Product

Resources

About