Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

Zheng, Kun; Ci, Kangyi; Li, Hui; Shao, Lei; Sun, Guangmin; Liu, Junhua; Cui, Jinling

doi:10.1016/j.bspc.2022.103609

Cited by 21 publications

(5 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The emergence of novel coronavirus pneumonia, COVID-19, has garnered worldwide attention. To address the low-performance problem in conventional methods caused by the lack of facial information, particularly when wearing masks, Zheng et al [57] proposed a nonend-to-end CNN-based residual network model. The proposed model utilizes the location of human eyeballs to locate the frontal ROI, generates spatio-temporal feature images, and determines their authenticity.…”

Section: Signal Extractionmentioning

confidence: 99%

“…In order to increase the number of possible application scenarios when only a portion of the face is accessible, such as when a face mask is worn, a method must be able to manage diverse input skin areas. For example, Zheng et al [57] proposed a non-end-to-end CNN-based residual network model tailored to the case of face mask use. In DeeprPPG [83], a hybrid DL method with spatio-temporal convolutional networks was trained to accept video clips of various input skin regions as input for HR estimation.…”

Section: Gaps and Influencing Challengesmentioning

confidence: 99%

See 1 more Smart Citation

Review on remote heart rate measurements using photoplethysmography

Lee,

Sivakumar,

Lim

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Remote photoplethysmography (rPPG) gains recent great interest due to its potential in contactless heart rate measurement using consumer-level cameras. This paper presents a detailed review of rPPG measurement using computer vision and deep learning techniques for heart rate estimation. Several common gaps and difficulties of rPPG development are highlighted for the feasibility study in real-world applications. Numerous computer vision and deep learning methods are reviewed to mitigate crucial issues such as motion artifact and illumination variation. In comparison, deep learning approaches are proven more accurate than conventional computer vision methods due to their adaptive pattern learning and generalization characteristics. An increasing trend of applying deep learning techniques in rPPG can improve effective heart rate estimation and artifact removal. To consider more realistic disturbances into account, additional vital signs and large training datasets are crucial to improve the accuracy of heart rate estimations. By taking the benefit of contactless and accurate estimation, the application of rPPG can be greatly adopted in real-world activities, especially in precision sports.

show abstract

Section: Signal Extractionmentioning

confidence: 99%

Section: Gaps and Influencing Challengesmentioning

confidence: 99%

Review on remote heart rate measurements using photoplethysmography

Lee,

Sivakumar,

Lim

2023

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…One of the major reasons researchers used this library was due to its availability in the OpenCV. Researchers used a neural network-based classifier to detect ROI [100], [101], [102], a statistical model to match a person's face to an image frame [103] and different algorithms to track features points for head movement in image frames [104]. To cater to the movement problem and identify the correct ROI, one way would be to detect face and ROI for every single frame.…”

Section: A Face and Roi Detectionmentioning

confidence: 99%

Remote Photoplethysmography (rPPG): A State-of-the-Art Review

Pirzada,

Wilde,

Doherty

et al. 2023

Preprint

View full text Add to dashboard Cite

Peripheral oxygen saturation (SpO2) and heart rate (HR) are critical physiological measures that clinicians need to observe to decide on an emergency intervention. These measures are typically determined using a contact-based pulse oximeter. This approach may pose difficulties in many cases, such as with young children, patients with burnt or sensitive skin, cognitive impairments, and those undergoing certain medical procedures or severe illnesses. Remote Photoplethysmography (rPPG) allows for unobtrusive sensing of these vital signs in a variety of settings for health monitoring systems. Several research studies have been conducted to use rPPG for this purpose; however, there is still not a commercially available, clinically validated system that overcomes the concerns highlighted in this paper. We present a state-of-the-art review of rPPG-related research conducted including related processes and techniques, such as regions of interest (ROI) selection, extracting the raw signal, pre-processing data, applying noise reduction algorithms, Fast Fourier transforms (FFT), filtering and extracting these vital signs. Further, we present a detailed, critical evaluation of available rPPG systems. Limitations and future directions have also been identified to aid rPPG researchers in further advancing this field.

show abstract

“…We conducted experiments on three public datasets, COHFACE [56], MAHNOB-HCI [58], and UCLA rPPG [59]. To assess the accuracy of the rPPG pulse extraction algorithms, we employed evaluation metrics derived from recent publications [33,34,60]. These evaluation metrics mean absolute HR error (HR mae ), root mean squared HR error (HR rmse ), and Pearson's correlation coefficients (ρ).…”

Section: Datasets and Evaluation Metricsmentioning

confidence: 99%

Transformative Approach for Heart Rate Prediction from Face Videos Using Local and Global Multi-Head Self-Attention

Premkumar,

Anitha,

Danciulescu

et al. 2023

Technologies

View full text Add to dashboard Cite

Heart rate estimation from face videos is an emerging technology that offers numerous potential applications in healthcare and human–computer interaction. However, most of the existing approaches often overlook the importance of long-range spatiotemporal dependencies, which is essential for robust measurement of heart rate prediction. Additionally, they involve extensive pre-processing steps to enhance the prediction accuracy, resulting in high computational complexity. In this paper, we propose an innovative solution called LGTransPPG. This end-to-end transformer-based framework eliminates the need for pre-processing steps while achieving improved efficiency and accuracy. LGTransPPG incorporates local and global aggregation techniques to capture fine-grained facial features and contextual information. By leveraging the power of transformers, our framework can effectively model long-range dependencies and temporal dynamics, enhancing the heart rate prediction process. The proposed approach is evaluated on three publicly available datasets, demonstrating its robustness and generalizability. Furthermore, we achieved a high Pearson correlation coefficient (PCC) value of 0.88, indicating its superior efficiency and accuracy between the predicted and actual heart rate values.

show abstract

Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

Cited by 21 publications

References 53 publications

Review on remote heart rate measurements using photoplethysmography

Review on remote heart rate measurements using photoplethysmography

Remote Photoplethysmography (rPPG): A State-of-the-Art Review

Transformative Approach for Heart Rate Prediction from Face Videos Using Local and Global Multi-Head Self-Attention

Contact Info

Product

Resources

About