Improved motion robustness of remote-PPG by using the blood volume pulse signature

Abstract

Remote photoplethysmography (rPPG) enables contact-free monitoring of the blood volume pulse using a color camera. Essentially, it detects the minute optical absorption changes caused by blood volume variations in the skin. In this paper, we show that the different absorption spectra of arterial blood and bloodless skin cause the variations to occur along a very specific vector in a normalized RGB-space. The exact vector can be determined for a given light spectrum and for given transfer characteristics of the optical filters in the camera. We show that this 'signature' can be used to design an rPPG algorithm with a much better motion robustness than the recent methods based on blind source separation, and even better than the chrominance-based methods we published earlier. Using six videos recorded in a gym, with four subjects exercising on a range of fitness devices, we confirm the superior motion robustness of our newly proposed rPPG methods. A simple peak detector in the frequency domain returns the correct pulse-rate for 68% of total measurements compared to 60% for the best previous method, while the SNR of the pulse-signal improves from - 5 dB to - 4 dB. For a large population of 117 stationary subjects we prove that the accuracy is comparable to the best previous method, although the SNR of the pulse-signal drops from + 8.4 dB to + 7.6 dB. We expect the improved motion robustness to significantly widen the application scope of the rPPG-technique.

Algorithmic Principles of Remote PPG

“…Existing rPPG methods [2]- [5] (except 2SR [6]) use the spatially averaged RGB values of skin-pixels to generate temporal RGB-signals for pulse extraction. The spatial pixel averaging step can reduce the camera quantization error.…”

“…In contrast to the BSS-based methods that impose no assumption on the colors associated with the source-signals, the model-based methods [4], [5] use knowledge of the color vectors of the different components to control the de-mixing. Therefore, these methods have one step in common: eliminating the dependency of C(t) on the average skin reflection color (i.e., DC-level), including the light source color and intrinsic skin color.…”

“…In recent years, several core rPPG methods have been proposed for extracting the pulse-signal from a video. These include: (i) Blind Source Separation (e.g., PCA-based [2] and ICAbased [3]), which use different criteria to separate temporal RGB traces into uncorrelated or independent signal sources to retrieve the pulse; (ii) CHROM [4], which linearly combines the chrominance-signals by assuming a standardized skincolor to white-balance the images; (iii) PBV [5], which uses the signature of blood volume changes in different wavelengths to explicitly distinguish the pulse-induced color changes from motion noise in RGB measurements; and (iv) 2SR [6], which measures the temporal rotation of the spatial subspace of skinpixels for pulse extraction. The essential difference between these rPPG methods is in the way of combining RGB-signals into a pulse-signal.…”

Attacks on Heartbeat-Based Security Using Remote Photoplethysmography

“…• Blood-volume pulse vector (PBV) [18]: PBV uses a precalculated blood-volume pulse vector (a unit-length color vector) as a prior to compute the weights for extracting the pulse from the RGB color-channels and suppress the (motion-induced) distortions.…”

“…To this end, we implement three stateof-the-art rPPG algorithms called CHROM [13], PBV [18] and 2SR [19], and evaluate how accurately these algorithms can obtain IPIs compared to a reference contact PPG (cPPG) sensor, which is often considered as a trusted device in related work [2], [4], [5]. We consider a wide range of settings and parameters that are expected to affect the accuracy of rPPG (and thus security) in practice, including subjects with different skin tones, different light sources, types and intensities of movement, video compression and camera-frame rate.…”