Contrast-Phys: Unsupervised Video-Based Remote Physiological Measurement via Spatiotemporal Contrast

Abstract

Video-based remote physiological measurement utilizes face videos to measure the blood volume change signal, which is also called remote photoplethysmography (rPPG). Supervised methods for rPPG measurements achieve state-of-the-art performance. However, supervised rPPG methods require face videos and ground truth physiological signals for model training. In this paper, we propose an unsupervised rPPG measurement method that does not require ground truth signals for training. We use a 3DCNN model to generate multiple rPPG signals from each video in different spatiotemporal locations and train the model with a contrastive loss where rPPG signals from the same video are pulled together while those from different videos are pushed away. We test on five public datasets, including RGB videos and NIR videos. The results show that our method outperforms the previous unsupervised baseline and achieves accuracies very close to the current best supervised rPPG methods on all five datasets. Furthermore, we also demonstrate that our approach can run at a much faster speed and is more robust to noises than the previous unsupervised baseline. Our code is available at https://github.com/zhaodongsun/contrast-phys.

Full-Body Cardiovascular Sensing with Remote Photoplethysmography

“…Several unsupervised approaches have been introduced to reduce the need for simultaneous PPG ground truth. Most approaches leverage contrastive training strategies, where similar pairs of samples are pulled closer and different samples are repelled [10,40,43,48]. The first non-contrastive approach leverages strong periodic priors to encourage the model to predict sparse signals in the frequency domain [38].…”

Heart Rate and Oxygen Saturation Estimation from Facial Video with Multimodal Physiological Data Generation

“…To overcome the problem of supervised methods, recent HR estimation methods focus on self-supervised (or unsupervised) learning approaches [23]- [27]. Specifically, they only utilize facial videos without supervised labels to train the models.…”

“…Specifically, they only utilize facial videos without supervised labels to train the models. These methods generate positive (i.e., similar HRs) and negative (i.e., dissimilar HRs) pairs by means of frequency resampler [23], frequency modulation [24], sparsity-based temporal augmentation [25], [26], and contrast between two different facial videos [27]. Contrastive learning is then applied to pull together the positive pairs and push away the negative pairs.…”

“…This presents a challenge because it is expensive to provide two different types of cameras for monitoring vital signs. Contrast-Phys [27] is a self-supervised 3DCNN-based method that predicts pulse waves from facial videos. It pushes away pulse waves predicted from two different videos (i.e., negative pairs) since the HRs of different videos are not similar, while pulse waves predicted from the same video in different spatiotemporal locations (i.e., positive pairs) are pulled together.…”

“…We build a specific model for each camera to prevent the estimation performance from being degraded by camera differences. Furthermore, unlike previous self-supervised approaches [23]- [27], CalibrationPhys also attempts to predict respiratory waves from facial video in a self-supervised manner.…”